Amazon Comprehend Medical
For a while now, I’ve been working on a fun project called SnapSetup, a React Native application that helps pharmacies set up their inventory easily. The app makes use of Google Vision and Amazon Comprehend Medical.
First, you take a picture of a drug package. The text on the picture is then extracted with the help of Google Vision’s OCR (optical character recognition.) Finally, with the help of Amazon Comprehend Medical, you get the generic name, brand name, dosage, form, and other information from the extracted text.
The lack of drug naming convention standardization is a problem, especially for those who are interested in creating a consolidated drug database. In most cases, drug names differ from pharmacy to pharmacy making it difficult for one to map pharmacy specific drug. One of the ways one could solve this issue is to bypass pharmacy specific information.
The goal of the SnapSetup is to provide pharmacies with a tool that standardizes the names of the drugs on their shelves.
Now for those who don’t know much about OCR and Comprehend Medical, I will give you a brief description of what they are.
OCR stands for optical character recognition. It is the mechanical and electrical conversion of handwritten, typewritten text into machine-encoded text. It is a common method of digitizing printed texts so that they can be electronically searched, stored, displayed, and used in machine processes such as machine translation, text-to-speech, and text mining. It can be used anywhere to process text without human intervention. In banks, it is used for validating checks, though the technology has not yet been perfected.
Google Vision is an API that allows developers to easily integrate vision detection features within applications, including image labeling, face, and landmark detection, OCR, and tagging of explicit content. OCR is just one part of Google Vision.
The Code
Here’s a snippet of code that performs OCR using Google Vision:
If you are using this, make sure you attach your own Google Cloud Vision API key to the URL like I’ve done. The Config you see in the code is a little library, react-native-config which helps pick environment variables from the .env file in the root folder.
ocr is the function that does the magic. It takes an argument base64, which should be the base 64 string conversion of the image you want to extract the text from. request is the parameter passed to the body of the API request. A request contains a list of images and features. The image contains a base64 string and features tells the API which functionalities you want to access. Feature with type TEXT_DETECTION tells the API it’s the OCR service you are requesting.
getText is a utility I wrote that helps get the string from the response.
Amazon Comprehend Medical is a natural language processing service that makes it easy to use machine learning to extract relevant medical information from unstructured text. Using Amazon Comprehend Medical, you can quickly and accurately gather information, such as medical condition, medication, dosage, strength, form, generic name, brand name, and frequency from a variety of sources like doctors’ notes, clinical trial reports, and patient health records.
Here’s the code that does the job
The code is pretty straight forward. acm is the Amazon Comprehend Medical function that does the drug name detection. It takes the text we got from our Google Vision API and extracts the generic name, brand name, form, dosage, etc. Depending on the version of Amazon SDK you are using you may have to import ComprehendMedical differently. That’s one of the issues I often encounter when I use the AWS SDK. In my current project, I used version^2.390.0, if you are using the same version you can go ahead and import exactly the same way I have done. Again Config is the library used for getting environment variables from the .env file in the root folder. I use Comprehend Medical by instantiating it and providing the necessary parameters ie. region, API version, access key id, and secret access key. You can read more on Comprehend from this link.
