Build a Go based REST API to query time series tweet data in Cassandra
One of the previous blogs demonstrated a time series app using which a continuous stream of tweets (high velocity data) is consumed and persisted to a Cassandra instance (using Oracle Data Hub Cloud) and is later queried using another service
Getting started with Cassandra using Oracle Data Hub Cloud
Build a Twitter data based time series app with Oracle Data Hub Cloud
Both the sample applications (i.e. the Tweets Producer service as well as the Tweet Query service) accompanying the blog post were Java based. In this blog post will, we will re-work the Twitter Query app to showcase a Golang based service (instead of the Java based Spring Boot app) and deploy it to Oracle Application Container Cloud (yes, it supports Golang!)
To be specific
Before moving on, I would recommend reading the following sections from the previous blog to get some context of the use case/solution (which is pretty simple!)
- Architecture overview
- Few words on the Cassandra data model
the app is available on Github
Contribute to accs-golang-cassandra-tweet-query-service development by creating an account on GitHub.
We will only explore the relevant details of our Go based Tweet Query service
Start by connecting to the Cassandra instance on Data Hub Cloud and creating a
Notice how the connection properties (Cassandra node/host and port as well as the credentials are picked up from the environment variables
DHCS_* — thanks to Service Bindings!
The core logic resides in the handlers — there are four of them (with self explanatory names)
getTweets function — it iterates over the bunch of tweets received from Cassandra, creates
tweetinfo objects and sends back a slice
tweetinfo is a simple struct with required attributes of the tweet
All these handlers follow the same pattern
- executes a
getTweetsto get a slice of
- and encode them as
jsonand return to the caller — using the native
the main method is the entry point to the app which brings everything together to start off the HTTP server
Service Binding to Data Hub Cloud
Application Container Cloud provides out-of-the-box Service Binding for Data Hub Cloud. This gives your app a secure communication channel without you having to do anything explicitly
no port related configuration is required at the database infrastructure level
Here is the documentation for this feature
Build & deployment
Before moving on, please go through the Infrastructure setup section of the previous blog to complete the following mandatory steps
- Setup a Cassandra cluster using Oracle Datahub Cloud console and bootstrap Cassandra (keyspace and table)
- Create a Twitter app which provides us with the required authentication tokens
Build the Tweet producer app
Start by fetching the project from Github —
git clone https://github.com/abhirockzz/accs-cassandra-twitter-timeseries-app
mvn clean install
The build process will create
accs-cassandra-tweets-producer-dist.zip in the
Build the Tweets query service
Start by fetching the project from Github —
git clone https://github.com/abhirockzz/accs-golang-cassandra-tweet-query-service
Zip up the following files —
Deployment a.k.a push to cloud
With Oracle Application Container Cloud, you have multiple options in terms of deploying your applications. This blog will leverage PSM CLI which is a powerful command line interface for managing Oracle Cloud services
You can download and setup PSM CLI on your machine (using
psm setup) — details here
Deploy both the applications
- Tweets producer
deployment.json with your Twitter access tokens and Oracle Data Hub Cloud instance details
“TWITTER_CONSUMER_KEY”: “<as per your app>”,
“TWITTER_CONSUMER_SECRET”: “<as per your app>”,
“TWITTER_ACCESS_TOKEN”: “<as per your app>”,
“TWITTER_ACCESS_TOKEN_SECRET”: “<as per your app>”,
“name”: “<as per your instance>”,
“username”: “<as per your instance>”,
“password”: “<as per your instance>”
Launch that zip into the cloud !
psm accs push -n TweetsProducer -r java -s hourly -m manifest.json -d deployment.json -p target/accs-cassandra-tweets-producer-dist.zip
- Tweet query service
deployment.json for this service as well
"name": "<name of your Cassandra instance>",
"username": "<Cassandra username>",
"password": "<Cassandra password>"
.. and deploy this app as well
psm accs push -n TweetsQueryService -r golang -s hourly -m manifest.json -d deployment.json -e dockerhub -p accs-golang-tweet-query-service.zip
Once executed, an asynchronous process is kicked off and the CLI returns its Job ID for you to track the application creation
After the apps are deployed, navigate to the Oracle Application Container cloud applications page to confirm — note down the application URLs
Test it out….
The way you would test this is already outlined in Test Drive section of the previous blog — no point repeating it. Here is the gist
- Start the tweets producer app and wait for the producer to start consuming tweets and persisting them to Cassandra
- Then use the REST API exposed by the Tweets Query service
- created a service to bombard a Cassandra instance on the cloud with Tweets!
- and then deployed a Go based REST API to interact with Cassandra to help you access those tweets in different ways
That’s all there is to it!
Don’t forget to…
- go through the Oracle Data Hub Cloud documentation for a deep dive
Oracle Data Hub Cloud Service — Get Started
Documentation that helps administrators, developers, and users get started using Oracle Event Hub Cloud Service.
- check out the tutorials for Oracle Application Container Cloud — there is something for every runtime!
Oracle Application Container Cloud Service — Create Your First Applications
Tutorials for Oracle Application Container Cloud Service. Learn to create your first applications.
- other blogs on Application Container Cloud
Latest stories and news about App Container Cloud — Medium
Read the latest writing about App Container Cloud. Every day, thousands of voices read, write, and share important…
The views expressed in this post are my own and do not necessarily reflect the views of Oracle.