Microservice to Support Amazon Web Services APIs
A microservice that provides unrestricted access to AWS APIs while maintaining a consistent and simple deployment process for DevOps across various regions within Amazon Web Services
VMware Workspace ONE Intelligence (WS1 Intelligence) is a cloud service built for the VMware Workspace ONE platform that provides deep insights, analytics, and automation for the entire digital workspace. WS1 Intelligence consists of about 30 JAVA-based microservices and 25 AWS Lambdas used in conjunction with many different AWS managed services (e.g., Kinesis, SQS, Dynamo DB, Redshift, Elasticsearch, Glue).
As one might imagine, functional and end-to-end tests for such a system can be a complex matter. It is often useful to be able to query the state of a specific service or datastore in order to verify that the system is functioning correctly. While this required information is generally available via AWS APIs or private APIs built by the team, these APIs are not generally exposed to external systems — in this case, the test automation client.
Providing special access to run automation code creates unnecessary overhead for DevOps and creates inconsistency in the firewall rules and infrastructure that is deployed across test and production environments. Fortunately, our deployment code naturally supports the construct of allowing certain microservices to be deployed only in pre-production (test) environments. This, for example, is a pattern that is followed every time we release a new service — the service goes into the test environments and, after sufficient development and test time, is added to the production footprint.
As a result, we addressed this issue by creating a new microservice just to facilitate automation testing. Automation tests utilize this microservice to make API calls to AWS or to private Intelligence APIs. DevOps can deploy this microservice in environments that run test automation thus eliminating the need to create special access roles and policies for specific user profiles within AWS IAM (Identity and Access Management).
This new microservice is called Trooper.
Trooper API
Test automation for the WS1 platform utilizes AWS APIs backed by Trooper, our new microservice. One such example is adding records to Kinesis streams. We are using the AWS SDK v2.
To be able to add records to the Kinesis stream, test automation code will require write access to that stream. Trooper has the required access, so the automation code needs to call the Trooper API that in turn invokes AWS API to perform stream operations.
We created custom domain objects to avoid breaking the Trooper APIs in case there are changes to the AWS SDK.
The following snippets demonstrate the usage for Trooper API to add records to Kinesis streams:
Trooper stream management service contains logic to:
1. Add records to Kinesis stream after mapping the Trooper request to AWS put records request
2. Return the Trooper response after mapping the AWS put records response retrieved by executing the AWS API to add records to the stream
Trooper Service
Trooper Controller
Trooper Stream API to add records to the Kinesis stream
Trooper Service API call
A rest client for Trooper service is created using Spring’s rest template.
TrooperResponse should contain the failed record count, if any, along with the shard ID of the Kinesis stream, error code, and error message.
We have created similar APIs within Trooper for DynamoDB, S3, etc. This has simplified our test set up and the automation code uses a single microservice for all AWS API calls thus eliminating the need to provide customized access and permissions to various components within AWS.
Service-to-Service APIs
Trooper is deployed along with the rest of the WS1 Intelligence microservices in AWS environments that run test automation. This deployment structure provides the flexibility to execute internal APIs (that require service-to-service authentication) via Trooper. Thus, we do not need to create new APIs (in this case, System APIs that require basic authentication) just for test automation.
Creating Trooper has reduced overhead for DevOps and it has greatly benefited our test automation framework by providing a single rest client to make API calls to AWS components, and to rest of the internal microservices within the WS1 Intelligence platform.
