NLP Chat on Flutter + Azure
Walkthru the design considerations of a full stack application!
UX Design Goals:
- Basic Chat App built in Flutter because Flutter is cool [1]
- Support 1-on-1 and Group Chat rooms
- Each room will have a realtime Sentiment score which is the collection of all chats (or last n-chats)
- Each individual chat will also have a realtime sentiment scoring applied w/ emoji to represent score
System Design:
High Level Design Details
- GUI has realtime push messaging using Google Firestore Cloud Messaging
- Messages are persisted in Google Firestore Cloud Storage
- Python REST API server performs Sentiment Analysis
- NLP Sentiment Analysis is based on BERT models with fine-tuning
- NLP inference engine is hosted on Azure ML Containers
System Design Considerations
1. Flutter vs ReactNative vs Native (or Xamarin)?
At this point Flutter has alot of momentum and works really well to build cross-platform apps quickly and has web support as well. On the downside the community & 3rd party support is in its infancy and the platform is changing fast. Obvious choice to use the newest and coolest tech now that its past Beta.
2. Realtime Notification HUB
I wanted realtime push of messages rather than polling. Unfortunately once you choose Flutter, it has a ripple effect on what push hubs you can use. Flutter aligns with Googles Firebase and has near native support for Flutter Cloud Messaging (FCM). Integration with Azure Events does not seem straightforward hence the quick path is to use Firebase for both Events and Storage.[11]
3. Hosting & NLP Services
Given Flutter and Firebase use, the natural choice is to use Google Cloud Platform rather than Azure for all hosting. I chose Azure for personal reasons — being more experienced on it, and understanding the Azure Machine Learning environment better. I am going very “polyglot” and going with a Python Flask REST to host Python ML models — very cost effectiveness of way to host services on Azure (esp if you bundle onto an existing web app — plan B is to use Azure Functions).
There are various NLP Sentiment Analysis [2,3,4] models you can implement. The cutting edge at this point is Google’s BERT and its variants[6,7]. Microsoft has released Azure Compute compatible pre-trained models which we can fine tune [8]. It maybe worth building a few models, starting with simple ones so we can measure differences/improvements.
This was a very brief article to kickoff my new project and explain how we make some basic design decisions up front. Look for Part 2 code + walkthru as I build it soon!
References and Inspirations:
[1] Flutter Overview — https://medium.com/swlh/why-businesses-should-start-focusing-on-googles-flutter-and-fuchsia-48e16820f2a9
[2] Nice Overview of NLP w/ code— https://towardsdatascience.com/a-practitioners-guide-to-natural-language-processing-part-i-processing-understanding-text-9f4abfd13e72
[3] Guide to word embeddings — https://towardsdatascience.com/beyond-word-embeddings-part-2-word-vectors-nlp-modeling-from-bow-to-bert-4ebd4711d0ec
[4] More on Sentiment Analysis — https://towardsdatascience.com/sentiment-analysis-concept-analysis-and-applications-6c94d6f58c17
[5] Azure already has a Sentiment service — https://azure.microsoft.com/en-us/services/cognitive-services/text-analytics/?WT.mc_id=blog-medium-abornst
[6] BERT primer — https://towardsdatascience.com/bert-explained-state-of-the-art-language-model-for-nlp-f8b21a9b6270
[7] More on BERT — https://medium.com/sciforce/googles-bert-changing-the-nlp-landscape-5f4a7bf65cc5
[8] Azure and BERT — https://azure.microsoft.com/en-us/blog/microsoft-makes-it-easier-to-build-popular-language-representation-model-bert-at-large-scale/
[9] Github details on pre-training and fine tuning BERT — https://github.com/google-research/bert
[10] Github for Azure BERT implementations — https://github.com/microsoft/AzureML-BERT
[11] Flutter and Firebase integration — https://medium.com/flutterpub/enabling-firebase-cloud-messaging-push-notifications-with-flutter-39b08f2ed723
