Validating the concept: Creating an api to access the data
Introduction
This is the second post in our blog post series about validating, scaling and evolving a full stack web application utilizing state of the art technologies as managed services in the cloud, serverless and modern web frameworks.
To read more about the overall plan find the first post here.
Our initial implementation of a service which collects data about tweets containing some defined keywords is running for some weeks now. We gathered enough data to be able to work with it and see which insights we can generate.
As we described in our first blog post, we want to produce small increments, in order to always be able to validate our work and adjust our next steps accordingly. So for us the next rational step was to make sure the gathered data can be used for visualization in a chart. The goal of this increment was the possibility to compare the performance of two or more keywords regarding the amount of tweets published over the timeframe of one day.
Architecture
The data we need to access is persisted in DynamoDB. As we need to do some more or less calculation heavy mapping and aggregation of the tweets we have collected, we decided on introducing an API-Layer which can be accessed from the front end and provides ready to be used timeline data. Between the API-Gateway and the DynamboDB a simple Lambda is responsible for mapping the data into the needed format.
The front end bundle and its assets are stored in a S3-Bucket. If you search the web for how to make static content located inside of a S3-Bucket accessible from an users browser, you will find several approaches. One of them will be simply using the website functionality of S3. This generates an URL for you and makes the Bucket act as a web server for static content. In our case we need to be able to access an API from our front end application. As the API and the front end are located behind different hosts, the well known issue regarding CORS will occur, which we would need to handle. Also we would need a mechanism for providing the URL of the API to the front end, which would add complexity that has nothing to do with visualizing data about tweets in a chart. For this reason we decided to put the front end behind the same API-Gateway we use to access our Lambda. Therefore, you just create two resources in API-Gateway (e.g. “/” for serving the front end and “/api” for accessing the Lambda-Function). Using this approach our front end is able to use relative paths for back end calls and the CORS issue is solved as well.
Result
You can find the full code, which you can deploy your own in this repo.
Serverless applications don’t need to be big and complex. Also small usecases can be implemented in a serverless manner, creating a solid and extensible foundation for your application to scale.
What else is important, is that by thinking how to implement our use case the “Serverless-Way” we automatically have to think about how to seperate concerns. The Lambda knows nothing about how and when data is persisted into the DynamoDB and our front end knows nothing about the Lambda, it just knows it can fetch data from a relative path.
What’s next?
Our next increment was successfully delivered. We verified that the data we gathered before can be used to visualize relevant information. For validation of the initial concept we took some shortcuts like hardcoding the search terms. Now it’s time to identify what is needed in order to scale the application and make it production ready. So again, stay tuned for the next posts in this series!
This blogpost is published by Comsysto Reply GmbH
