New to Javascript based testing tools?
A case study of automation with Puppeteer.
Work with environment variables & APIs. Make sure you do not push secrets to VCS!
Technogise gave me an opportunity to work on a project which had very interesting requirements. It was a fun filled short journey with insightful learnings.
My responsibility was to automate a scenario — we needed to absorb some data from a third party application (TPA) and insert that information into our database for later use. Due to the lack of public APIs, we had to go the GUI route. Our preference was to use a JS tool.
What follows is my experience. Hope you enjoy the read !
Unknowns we overcame :
- Our team wanted to use Cypress initially. However, we chose Puppeteer later. Let’s read what happened there…
As we know, some websites do not load in iFrames (for security reasons). This TPA was one of them i.e. did not load in an iFrame.
We also know that Cypress loads its application under test (AUT) in an iFrame. Therefore, we could not use Cypress. So, we picked up Puppeteer after some investigation. - I encountered that our test could not run in headless mode. This TPA would not login unless we were running an actual browser. Our servers were Unix and had no GUI where this script would run. As a result, we opted to use Xvfb to run the script on our server.
- We were going to run this script almost daily for a large number of data points (roughly, tens of thousands of data points). Hence, I was testing for any rate limiting by the TPA or effective IP / account blocking. That never happened, but we observed that the website (TPA) started responding way too slowly after the script ran for a while. Initially, we were keeping the same user session for the entire run of the script. After some discussions, we decided to create a fresh user session. The way this worked was — get a data point from our system, go to the TPA and collect the rest of the data around it, store it in our DB and re-login after restarting the browser.
Handling environment variables :
We needed to work with some sensitive data like login information of the TPA in our code. Following security best practices we decided to use dotenv dependency.
Following is the usage example :
We would store this information in our zshrc or bashrc, rather than committing it to our codebase.
Dealing with API calls :
As I mentioned earlier, we needed to access our APIs to make some GET and POST requests.
Following are the code samples for the same :
Pre-commit hooks usage :
Keeping application security in mind, I added the following checks in our codebase:
- A scanner that will block any sensitive commits to the pipeline. I used Hawkeye for this purpose.
I added it as a devDependency and then provided following instructions in package.json
- An audit for dependencies to find any high severity vulnerabilities :
This makes sure that the commit does not go through if vulnerabilities are greater than or equal to a score of 8.
ES6 features :
- Used async methods :
and called it as :
- Used ES6 destructuring assignment :
and assigned them as follows :
- Running the test :
Notice here that the syntax is that of immediately invoked function.
Result :
A pretty neat experience with Puppeteer. Script execution is fast and reliable! Do give Puppeteer a try, it is very well documented.
