My SIOT Internship: Radar in Embedded Systems
During the summer of 2020, I had the opportunity to intern at GovTech’s Sensors and IoT (SIOT) Division, and worked on a proof of concept of detecting falls with cutting-edge 60GHz mmWave radar sensors. Ultimately, this was to experiment with fall detection to be potentially used within a Smart Home space where you might have seniors living alone or community-assisted living. Studies have shown that about 50% of seniors are unable to get up after a fall. Part of SIOT’s work in the GovTech Assisted Living Ecosystem (GALE) is to provide the seniors living alone with a panic button that they can use to quickly reach assistance. By incorporating fall detection into this ecosystem, we can further enhance the safety of our senior citizens.
Today, cameras coupled with machine learning algorithms are the primary means of advanced people detection. People detection with radars is interesting because it is a privacy-conserving means of detecting people within a space. Radars are also not susceptible to some challenges that cameras face, such as bad lighting conditions, sensitivity to weather conditions and inability to gauge speed accurately.
For my project, it had two major parts — implement people detection with a radar to determine where a person was within a space, and if they had fallen.
What is mmWave Radar?
60GHz mmWave radar is a new type of sensor technology pioneered by Texas Instruments, by incorporating the entire radar front end in CMOS architecture, enabling an SoC design combining the RF front end, DSP for I/Q signal processing, and the CPU for general instructions.
In principle, what this radar does is to send out a chirp — a specially modulated radio wave around its baseband frequency (60GHz) — and “listen” for the reflected signal as it bounces off nearby objects. In concept, it is akin to shouting into a large room and listening for the echoes to determine the size of the room. You can use the time delay between the broadcast and received pulses, as well as the beat frequency, sent through a Fast Fourier Transform (FFT) to determine the frequency of the received waves. This allows you to determine the Doppler and Signal-to-Noise ratio of the received signal, and subsequently, estimate position based on your radar’s coordinate axes.
This technology is commonly used in Autonomous Driving as an all-weather solution to detect surrounding obstacles. Thanks to the active nature of emitting its own sensory pulses, it can function in all light conditions and is relatively weather agnostic. Along with cameras, this makes radar a popular solution in the Advanced driver-assistance systems (ADAS) sensor market.
System Architecture
Interfacing with the IWR6843ISK radar module from TI was relatively straightforward using the standard UART interface: 8 data bits, no parity bit, and 1 stop bit. This allowed me to design and fabricate a custom PCB to control the radar module from an MCU and pipe out the required data to a server for subsequent analysis and display on a dashboard.
As is common with any project in SIOT, there is close integration between software and hardware. I worked closely with my colleagues to pick up key skills in hardware design and understand how design decisions relate to the eventual firmware.
Firmware
Initial testing was carried out in Python with a PC. When it came to the final firmware, it was written in C/C++ as this was required by the vendor toolchain for the MCU. I opted to write my code on FreeRTOS in a modular style — increasing visibility, and maintainability. However, this made debugging more difficult when optimising memory usage with the high data throughput from the radar sensor. After unit testing on a testbench in the lab, we were able to carry out field testing at the final deployment site, where we fine-tuned the sensitivity parameters of the radar to increase accuracy.
People Detection Deployment
While on-site, we realised that the performance varied drastically with each environment. We had to manually tune the sensor to maximise accuracy due to obstructions and the nature of targets that were passing through the scene. Thanks to the help provided by our project partners, the installation was a breeze! Power and data lines were easily set up and our weatherproof enclosure endured the rainy season admirably, allowing us to validate the device functionality in rainy conditions. We leveraged SIOT’s DECADA network to easily pipe data back to our own servers.
Machine Learning for Fall Detection
Since fall detection was still in an experimental stage, we did not get a chance to deploy it in the field and testing was carried out in office.
I tested a fall detection algorithm using a Variational Recurrent Autoencoder in Tensorflow to classify incoming data as either fall, or not-fall. This model was chosen as a suitable unsupervised learning model for time-series data. The model was trained with in-house data of falls that I had to create (ouch!) and tested in the same environment However, the model was only as accurate as a simple height-based algorithm due to the significant noise in the incoming data. Testing was done with a larger training set but we could only make minor improvements.
Fundamentally, the issue was the amount of noise present in the radar sensor. It was difficult to identify consistent features that correlated with a fall, especially at a further distance from the radar sensor. Utilising the richer, raw I/Q data stream would have allowed us to identify the features in greater resolution, and provide a better feature set for fall detection.
The Internship Experience
Overall, my internship was a really fulfilling one! While the majority of it was under work-from-home conditions — due to the COVID-19 pandemic — my reporting officer, Mr Tan Ah Soon, helped me make the best of my situation and I learnt a lot from him. The team was friendly and approachable and often lent me a helping hand. My internship at SIOT also taught me some Software Engineering best practices that proved to be invaluable while working with larger teams. Git for version control, Confluence for documentation and unit testing for Continuous Integration / Continuous Development (CI/CD) are all important tools for a group of engineers to work efficiently as a team.
I really enjoyed working on the project because I not only got to work with the firmware of the radar, but I also got involved with the hardware design. Working with the radar was a challenge because it exposed me to a new field and I had to educate myself rapidly to get up to speed on the technologies behind radars. However, ultimately it was a rewarding experience because I got to learn a lot within a short space of time on a meaningful project that could go on to help senior citizens. I feel that this is typical of internships in GovTech: you are given a challenging project with the resources and freedom to solve the problem that ties back to a bigger picture.
Conclusion
If you are considering an internship with GovTech, I would highly recommend it! You stand to learn various technical and teamwork skills, and make a real impact to the project at hand.
~ Ming Jie
References
- Tinetti ME, Liu WL, Claus EB.Predictors and prognosis of inability to get up after falls among elderly persons. JAMA. 1993;269:65–70.
Find out more about GovTech’s technical products and solutions on Singapore Government Developer Portal (www.developer.tech.gov.sg)
