End to End Test Automation — SQL Server, Event Store, MongoDB, ElasticSearch, React
An idea on building the End to End test automation for the product with the components as below.
Components –
1. CRM — Researchers enter the data.
2. Microsoft SQL Server — 2 databases
a. CRMDB
b. UserIdentityDB
3. Scooper — Component that picks up data from the CRMDB and puts in the Event store
4. Subscriber — Component that are listening to Event Store, who pick up the data from the Event Store and push it into MongoDB and Elastic Search
5. Middle Tier — Services that pull the data from MongoDB and the Elastic Search and send it to the Front End in the form of Rest apis.
6. Front End — Data entered by the Researchers to be displayed.
Architecture –
1. Researchers enter the data in the CRM.
2. Data is thereby stored in Microsoft SQL Server Database — CRMDB
3. Dedicated table is created in the CRMDB mentioned in the Step 2, where in the data entered in the original schema of the CRMDB is stored in the format for Event Sourcing.
4. Scooper / Batch Scooper does the job of picking up the data from the Dedicated table from CRMDB in Step 3 and push it to the Event Store.
5. Subscribers pick up the data in the Event Store and push it to MongoDB and Elastic Search.
6. On the other end, there is a Service oriented architecture, where in each of the services fetches the data from the Elastic Search and provide the data to the front end in the form Rest apis.
7. Front end is a single page application which consumes the data sent by Rest apis and renders the page.
8. There is a dedicated Identity server, which manages the identity of the user across all the applications the user has access to as a part of the subscription. The database that holds details related to the user identity is present in the Microsoft SQL Server Database mentioned in Step 2.
Technology Stack –
1. Scooper / Subscriber / Services — .Net Core
2. Event Store — https://eventstore.org/
3. MongoDB — https://www.mongodb.com/
4. Elastic Search — https://www.elastic.co/
5. ReactJS — https://reactjs.org/
6. Amazon EC2 — https://aws.amazon.com/ec2/
7. Team City — https://www.jetbrains.com/teamcity/
Functional Testing Challenges –
1. Verifying the data integrity i.e. data entered in the CRM is correctly persisted in the Elastic Search and MongoDB.
2. Verifying if the data present in the Elastic Search & MongoDB is correctly displayed on the Front End.
End to End Test automation –
Pre-requisites –
Export the rows from the table in the CRMDB where in data is stored in the Event Sourcing format in Sql file.
Automation workflow –
1. Delete existing rows from table in the CRMDB where in data is stored in the Event Sourcing format.
2. Delete the events from Event Store.
3. Clear the MongoDB and the Elastic Search.
4. Apply the latest schema to the CRMDB and the updated Stored Procedures.
5. Insert the Sql files in Pre-requisites (Step 1) in the table in the CRMDB where in data is stored in the Event Sourcing format.
6. As the events start flowing through the Event Store, verify if none of the Events are going in the Event Store — Parked Queue.
7. Verify if none of the events in Event Store are InFlight.
8. Wait till the time, all the rows from table in the CRMDB where in data is stored in the Event Sourcing format and the events in the Event Store are 0.
9. Trigger the execution of test cases.
Tests –
Data integrity tests — These tests verify if the data from the table in the CRMDB where in data is stored in the Event Sourcing format has passed through the intermediate components Scooper, Event Store and Subscribers and has reached MongoDB and Elastic Search.
Api tests — These tests verify if the api endpoints are sending the data in the format expected by the Front End.
UI tests — These tests are the Functional tests that verify the business use cases.
Automation Technology Stack –
Api automation –
b. https://www.npmjs.com/package/supertest
UI automation –
