We, Pahan, Chathuni and Pamuditha, are three undergraduate students from Sri Lanka, a small island in the Indian Ocean. But we were never ones to limit our ambitions. We had hopes to compete in the InnovateFPGA Design Contest, a prestigious competition organized by Terasic Inc. and Intel Cooperation. But, winning an award, even at the regional level, didn’t seem easy at first. Nevertheless, we wanted to get out of our comfort zones and do our best in something that piqued our interest. This is our story in the InnovateFPGA contest, how we began and progressed to win the world’s Silver Award!
Content
- What is InnovateFPGA?
- How We Started
- Our Project
- Regional Semi-finals
- Regional Finals
- Grand Finals
- Task Delegation
- Ending Thoughts
What is InnovateFPGA?
First of all, let me tell you what InnovateFPGA Design Contest is!
InnovateFPGA is a global competition open to anyone who is interested in innovating using Intel FPGAs. This is organized by Terasic Inc. with Intel Cooperation as the headline partner, Analog Devices and Microsoft as diamond partners along with Mouser Electronics, Arrow Electronics, Digi-Key and Macnica as gold partners. As you see, the industrial exposure that can gained through this competition is huge.
The organizers introduce a new theme and a new platform for the competitors to utilize for each year. This year, it was all about connecting the edge with the cloud for a sustainable future. To achieve this, they have partnered with Intel, Analog Devices and Microsoft to provide the platform with all the necessary means to come up with powerful and versatile solutions.
Microsoft Azure IoT Edge, IoT Hub, IoT Central
The competition spans for nearly a year. It was launched back in July 2021 and progressed through several stages until June 2022.
We can win incredible cash prizes too!
Now let’s get into the inside story of our journey in the InnovateFPGA!
How we started
Our interests in participating in the InnovateFPGA Design Contest started growing some time ago. We were inspired by the achievement some of our seniors had in the regional level of the competition. So, we dreamed…
Around the start of September, we received news that the proposal submission for this year’s competition has opened. Even though we hoped to participate one day, we thought this is not the time, as we had started another project for the International Future Energy Challenge(IFEC). Deadline for the proposal submission passed but we later heard that it has been extended for another month, specially for SDG projects. The final deadline for the proposal submission of both the InnovateFPGA and IFEC was the same day. But, we were able to finish our proposal for the IFEC around a week earlier. Also, we didn’t actually have any guarantee that our proposal for that would get selected as no previous teams from Sri Lanka had managed to do so. More about that in another post, dedicated for IFEC.
So, we decided to give InnovateFPGA a chance. But, we only had a week to do what the other teams had months to do. We invested the most of our time to brainstorming and coming up with a project idea.
In the end, after much brainstorming and also with the help of Kithmin Aiya, we decided to propose the Green Machine, a mini-greenhouse management system suitable for urban households but not limited to that. This project idea aligned well with the competition theme which was Sustainability. The key features that would differentiate our design from others in the market, was its modularity and scalability. Therefore, our design would be applicable in small to large scale farms and for research purposes as well. We registered our team in the InnovateFPGA page and wrote, what we believe to be a very convincing and interesting proposal. Read it here!
And we got accepted into the regional semi-finals along with 75 teams in the APJ region out of nearly 150 teams! That was huge news! In the meantime, our proposal for the IFEC got accepted too. We had a challenge to manage two projects, equally important, in parallel.
The next stage was to develop the mini-greenhouse management system that we proposed. For this, we received the following tools from the contest organizers and partners.
- Terasic Cloud Connectivity Kit; includes DE10 Nano development board and RFS daughter card
- EVAL-ARDZ-CN0398 for soil monitoring by Analog Devices
- DC2420A QuikEval Board for ADC by Analog Devices
- DC1338B QuikEval Board for power analysis by Analog Devices
Then, the real work started.
About the Project
Now, I will explain about about how we defined and developed our project.
From the beginning, we wanted to make our solution have the following characteristics.
- Promotes sustainable agriculture in a wider domain
- Easily adaptable
- Affordable
- Scalable and Modular
- Highly optimized
- Expandable
To achieve these objectives, we came up with the concept of the Green Machine which is a smart IoT mini greenhouse that combines the power of edge and cloud computing. It can provide automatic watering, ventilation and light to the plants grown within it and uses a comprehensive set of sensors for monitoring. The highlight is that a single FPGA can control multiple units of greenhouses within a vicinity.
And the DE10 Nano we received contains not just an FPGA, but an SoC; an combination of FPGA and a HPS(Hard Processor System). This opened up many new paths for us.
The System Operation
The data related to the condition within the greenhouse is obtained using the following set of sensors and cameras.
- ArduCam Mini 2MP Camera
- Light Intensity sensor
- Temperature and Humidity sensor
- CO2 sensor
- Soil Moisture sensor
- Soil pH sensor
This data is collected by a microcontroller and sent to the DE10 Nano board over Bluetooth. The advantage of using Bluetooth was that its lower consumption than Wi-Fi over a shorter range. But, since the RFS module that provides the communication capability to the DE10 Nano also has Wi-Fi capabilities, we can switch to Wi-Fi if we want to expand our range. We implemented a UART module on the FPGA to provide the interface between the SoC and the RFS card.
Then, the received image and sensor data are processed on the SoC. Here, we utilized the power of the heterogeneous system on SoC to implement a neural network to detect abnormalities in the plants using the images. This was implemented using OpenCL and accelerated by the FPGA. It could classify the images into six categories up to an accuracy of 95%.
Then, the main control algorithm decides the amount of water, ventilation and light based on the CNN results and the sensor data. This is implemented on the processor of SoC and written in C++. Then, the instructions are sent to the greenhouses over the same Bluetooth link. Meanwhile, all the received data and the decisions taken are stored in the local memory and sent to the Microsoft Azure cloud periodically. They are stored in a CosmosDB database system and analyzed in the long run to identify trends and improve the system.
The main challenge we had during this implementation was creating a board support package that includes all three of OpenCL neural network implementation, cloud connectivity and UART communication. We could only find individual resources available for these three components without much guidance into how we can combine them. We could find a GitHub repository where they explained how we can modify the OpenCL board support package provided by Terasic to include custom IP blocks. By following this, we integrated the UART module into the BSP and used the network capabilities of the HPS to connect to the cloud.
Regional Semi-Finals
During the period of 5 months from November 2021 to March 2022, we implemented this project. We created a prototype of a greenhouse like this to demonstrate our project. We mainly focused on making this design attractive and easy to use.
For the regional semi-final round, we had to submit a demo-video and a design paper of our project by 3rd of April. During the project development phase, there were times we felt like we wouldn’t be able to present our project for the regional semi-finals at all. One time, after we got the structure of the greenhouse made and filled it with soil, it fell apart, unable to withstand the pressure of the soil on its four legs. It was like the whole world collapsing for us. But we didn’t give up. We got a trolley made, which can support the weight and even move the green house around.
In the end, we completed the design paper and demo video for the regional semi-final round.
Regional Finals
With this, we progress on to the regional finals with 11 teams in the APJ region. We were incredibly happy! For the regional finals we submitted our codes in a GitHub repository and faced a QnA session with judges from Terasic, Intel, Analog Devices and Microsoft. It was an excellent experience to interact with these prestigious judges and hear their valuable feedback!
And, we emerged as the Gold Award winners of the Asia Pacific and Japan region! This was unbelievable for us! We are the very first undergraduate team from Sri Lanka to win this. With this, we were invited to the Grand Finals in the Intel Innovation Campus in San Jose in June 2022 along with 10 other teams from all over world. All of these projects were incredible, so, we had a challenge.
Grand Finals
We were so excited. We were going to the Intel Innovation Campus for the Grand Finals, a trip fully sponsored by Terasic and Intel. Since we had to take our project prototype to present at the finals, we decided to create a smaller version of the greenhouse, which can be dismantled and moved.
For the grand final round, we created a custom PCB to connect the microcontroller, AD board and all the sensors.
Unfortunately, we received news that the Grand Finals had been shifted into a virtual format due to the rising corona cases in California. We were asked to create two videos, a demo video and a presentation, for the final competition. We received a webcam, a microphone and a ring light to shoot these videos. However, we used our own equipment, where Pahan utilized his skills in photography and videography, to shoot these video in studio quality. We received incredible feedback for our videos during the Grand Finals.
View these video here!
During the final competition, which was held on 1st of July, 2022, we presented our videos and faced a QnA session with judges from Terasic, Intel, Analog Devices, Microsoft and Digi-key. We received the highest enthusiasm from the judges. They were very interested to know more about our project and we received incredible feedback for the wide applicability of our project and its effect in sustainable agriculture.
And, we emerged as the Silver Award winner of the whole world for the first time in Sri Lanka.
Task Delegation
All three of us worked on coming up with the conceptual design of the Green Machine and its physical design. Pahan and Chathuni together assembled the Green Machines, installing the modules, sensors and actuators in optimal positions.
Pahan developed the UART system design on the FPGA and the hardware interface between the HPS and the FPGA through AXI. He also designed the PCBs and did the amazing videography for the project videos.
Chathuni worked on implementing the control system on the HPS and interfacing the processor with the UART on FPGA through the code. She also carried out OpenCL kernel development and compilation and Arduino programming in the Green Machine units.
Furthermore, Pamuditha designed and implemented the neural network. The 3D design of the Green Machine units was carried out by Pamuditha as well. He also undertook programming the controller in C++ and OpenCL host program development and CNN parameter organization.
The final system design for the Finals, that integrates the OpenCL implementation with the UART and provides a Linux kernel, was done by the combined effort and contribution of all three of us.
Thoughts
Overall, it was an incredible experience to take part in the InnovateFPGA Design Contest. We got to know many people from these tech giant companies and had the opportunity to share our ideas with them. Also, we were able to push our boundaries and a learn a great deal of things about SoC development and product design. Looking back, even we are surprised about how we managed two such projects in parallel and do an impressive job on both of them. If we never tried or never kept going, we would never know how much we are capable of.
We would like to thank Dr.Ajith Pasqual, Dr.Ranga Rodrigo, Mr.Abarajithan Gnaneswaran and Mr.Kithmin Wickramasinghe for all of their support and brilliant ideas that shaped our project into what it is now.
And, we are forever grateful to our parents who provided all the financial support to implement this project and for having our back when we were discouraged.
As the ending note, I would like to mention the one most important thing that we learned out of this.
Never be afraid to dream big!