The Work We Do (in the Cloud)
Building a data pipeline based on lamba architecture.
It has been more than 2 years since I first joined Data Studio (in Advanced Technology Center in the Philippines or ATCP) and since August 2018, the Data Studio have brought to life some amazing ideas that we now share to some of the clients and colleagues in Accenture who visit us in the Data Studio.
I would like to share one of the first demo applications that the team built on AWS. Kindly note that I am presenting this from a data technology perspective although I will try to share some other use cases where this can be applied. This is with the intention to spur new and better ideas from you.
Data Supply Chain: Processing & Leveraging Streaming data in Different Context
The demo app showcases the different context on how streaming data can be used. Because streaming data can be ingested as it is created, there is a tremendous opportunity to process this data in near real time and then store that same data for later analysis.
We built near real time and batch data processing demo application using AWS ‘server-less’ technologies.
The business use case where we applied this demo app was in a healthcare facility. In this facility, we have Tom, a 58 year old male who wish to live the best years of his life and then there’s Mitch, who cares for Tom and is responsible for Tom’s health and well-being.
Tom was given a wearable device that tracks his heart rate every minute. The data from the device is being ingested via AWS Kinesis Firehose and this streaming data can then go to a Speed layer and also to a Batch layer before being “served” in the Serving layer. This is shown in the diagram below.
In the Speed layer, the streaming data can be analyzed and processed as it is being ingested and any “abnormal” heart rate can be captured and can either be stored in AWS S3 or can be used in AWS Lambda function to trigger an event. In this use case, the app sends a notification (either email or SMS) to Mitch (using AWS SNS).
The same streaming data that is stored in AWS S3 can used for analysis via the Serving layer. In this layer, historical analysis of Tom’s heart rate can be made and Mitch will be enabled to see patterns (if there are any) to be a better healthcare partner for Tom.
The diagram shown above is one of the common architecture pattern called Lambda architecture. I would like to note that the name of the architecture pattern has nothing to do with the use of AWS Lambda service.
The above architecture leveraged ‘server-less’ technologies in AWS.
How data adds value in different context?
Now, going back to our use case. Mitch’s (the caregiver) ability to give timely care to Tom has been augmented and enhanced because he can now respond to any abnormal heart rate in this case.
The video below shows how Mitch can be notified — either via email or via SMS message to his phone.
In addition and as previously mentioned, Mitch is enabled to see patterns (if there are any) via dashboards as shown in the video below. In here, Mitch has access to Tom’s historical heart rate data, allowing him the ability to analyze and see patterns.
In our use case, he sees that there are certain and recurring times of the day when Mitch has a high occurrence of abnormal heart rate. With this insight, Mitch talks to Tom and he discovered that Tom is doing some strenuous activity during these times of the day so advises Tom to reduce on these activities.
How about the other use cases?
I know I promised that I will share other use cases where this solution can be applied. For this, here are some I can think of and are already solutions you may have seen and used.
Location-based solutions
Imagine being in your favorite shopping mall and getting notified that you might want to buy that perfume for your wife because her birthday is in next two days. The same notification provides you a summary of your previous purchases from the store and reminds you that you can use points you have earned from your previous purchases.
This solution can also be used in a hotel or a casino or a cruise ship where you want to enhance your customer’s experience by giving them relevant and non-intrusive recommendation or just help them navigate through the building or cruise ship and get timely help from staff members.
IoT-enabled appliances
Your air-conditioning unit (ACU) can notify you when it needs to be cleaned or when certain parts need repair. This same appliance can provide you information when the last cleaning was done and which service center did it. It can also inform you how much will be your expected electric bill based on your usage of your ACU.
Video security systems
Video surveillance systems are very prevalent nowadays but a human person cannot just watch a monitor all day. Video data can be analyzed as it is being captured and using machine learning or deep learning, any suspicious activity can be captured and relevant authorities can be alerted in near real time.
I am pretty sure you can think of other use cases and I hope you do. Let us know what you have in mind and we’ll see what we can work on together in the ATCP Data Studio.
Disclaimer: All views expressed on this story are my own and do not represent the opinions and viewpoints of any entity or organization that I have been, am now, or will be affiliated.
This story has been published for information and illustrative purposes only and is not intended to serve as advice of any nature whatsoever. The information contained and the references made in this story is in good faith, neither my employer nor its any of its directors, agents or employees give any warranty of accuracy (whether expressed or implied), nor accepts any liability as a result of reliance upon the information including (but not limited) content advice, statement or opinion contained in this paper.
This story also contains certain information available in public domain, created and maintained by private and public organizations. I do not control nor guarantee the accuracy, relevance, timeliness or completeness of such information. This story constitutes a view as on the date of publication and is subject to change.
This story makes only a descriptive reference to trademarks that may be owned by others. The use of such trademarks herein is not an assertion of ownership of such trademarks by me or my employer nor is there any claim made to these trademarks and is not intended to represent or imply the existence of an association between me and the lawful owners of such trademarks.
