Creating NLP driven risk dashboard for financial services in AWS
Real time risk dashboard is a crucial element of monitoring and insights in financial services industry. Due to dynamic nature of risk profiles across asset classes and products, creating a real time dashboard is increasingly becoming complex. Traditional dashboards, provides crucial information using analytics and charts. However, as the complexity increases, using such dashboard are becoming less user friendly. There is need to enhance such dashboard to provide a simpler conversational interaction using machine learning and natural language processing.
Such a system should be data driven with predetermined structure to news headlines to ensure business familiarity for users. Providing stronger search algorithms to enable users to get right event information is very crucial for actional insights. Designing such a system brings together several latent needs of more user friendly and personalized language driven dashboard in low latency scenarios. Some of the critical features of such an system are:
§ Leveraging advanced language processing capabilities to create right “news style description from tabular data”
§ Ensure that the news engine generates the most relevant news headlines and narratives for each user
§ Deliver the narratives in the most appropriate way and allow users to take actions against the narratives
§ Narratives are specified in common business driven dimesons an provides appropriate levels, type, sentiments, recency, and aggregations.
Given AWS comprehensive data analytics and machine learning offerings, creating such a dashboard can be relatively easy to achieve. It goes without the mention that a news centric language driven dashboard is architecturally complex. Nevertheless, AWS stack provides key building blocks and technology choices to build a highly availabile and low latency solutions.
As we move forward in building a system, the following diagrams provides our current thinking around system architecture.
Key feature of this architecture blueprints are as follows:
1. Ensuring that relevant data from applications, transaction and risk model systems are captured. Given the nature of such data, having scale analytics platforms such as EMR with its extended capabilities is one of the best in the current technology ecosystem.
2. Flexibility provided by Sagemaker for bringing your own container and models along with training computational scalability suites the needs for such high data volume and complex NLG centric language models training
3. Model deployment using EKS meets the need for scale serving at the run time for high frequency risk events updates
4. Keeping in line with real time data updates and refresh, Kinesis or MSK has ability to meet required latency
5. We have introduced Apache Ignite as architecture block as its performs at the need scale for news refresh and low latency across geo distribution and user profiles
Simplification of real time dashboard from user perspective is very ripe for change now. Power of NLG and language processing augmented with AWS enabled technologies provide such an opportunity for realizing this need.