An Attempt at Facial Recognition and Automatic Vitals Capture for Bahmni
Vision
We at ThoughtWorks Hyderabad, wanted to develop something that combines Machine Learning with IOT devices. We discussed internally and came up with an idea to find if we can build some automated yet cost effective solution to take vital measurements for Bahmni — the open source hospital system.
With this in mind, we started playing with Arduino and a bunch of physical sensors that could measure body parameters.
We got a couple of sensors from the local electronics market like `LM35`, `AD8232`, `Pulse sensor` that can measure temperature, ECG and pulse rate. Once we got these sensors working, we decided to integrate them intelligently with the Bahmni ecosystem.
All this while, we also explored face detection and face recognition in parallel. We played with `OpenCV` and found that `Local Binary Pattern Histogram` algorithm works best for real-time face detection/recognition.
Our code was recognizing faces as we wanted it to.
The next step was to merge the pieces together to have an end-to-end working solution.
Current progress
We set up a Python server on the machine with an in-built camera or external webcam. As part of the patient registration process in Bahmni, this server is triggered to open up the camera and train for patient faces. The server then stores the patient uuid (unique identifier) in the local database.
Once, this is done, and the patient is registered, (s)he can visit again anytime to have their basic readings taken automatically. The sensors would continuously read the values and on finding a valid input would trigger the face detection algorithm which uses the camera to identify the patient. The server then fires a Bahmni API to save the observation value for the patient.
As of now, the above setup works for temperature measurement for a patient.
Future plans
We are looking at building a ECG sensor that can take the readings from the patient’s thumbs. We are also working on an OCR (Optical Character recognition) tool that can take photos of already existing patient reports and convert them into digital records.
Challenges
- The sensors we chose currently are for used for research purposes only and not for medicinal use. Therefore, finding and integrating with efficient yet cheaper sensors to reduce cost is one of the main open challenges. The more robust and better insulated the hardware, the better it will be a fit for Bahmni.
- We also would need to merge face detection data on the main server from different local servers to have a unified database.
On a funny note, here is what happens when Engineers are given tools that they don’t understand : )
Thanks for reading.
Disclaimer: The views and opinions expressed by the author are personal to the author and do not necessarily reflect the positions of ThoughtWorks.
