How to retrieve data from GitHub in Cypress
When working on real-time projects, it’s crucial to retrieve data dynamically instead of hard-coding it. This practice is especially important in cases where data changes frequently, such as in an E-Commerce company where product prices are subject to monthly updates.
In these cases, reloading the file in your project with each update is not an efficient solution. As an automation engineer, you need to find an optimized solution.
One approach is to fetch the data from a GitHub repository whenever a developer updates it. So, instead of storing the data in your local code, you can pass the link to the repository and access the data dynamically during the validation step. This approach saves a lot of time, as you won’t have to update the data manually every time it changes.
However, getting Cypress to authenticate and fetch data from these repositories can be a bit tricky.
In this blog post, I will explain the steps to fetch data(CSV file) from your private GitHub repositories and use in it in the Cypress test.
Below are the steps involved in the approach:
1. Create a Personal Access Token in Github Profile
2. Form URL
3. Fetch data from GitHub Repository and write it to File in Cypress
Step 1: Create a Personal Access Token in Github Profile
For accessing a private GitHub Repository, you need to generate a token (Personal Access Token)
(i) Go to your GitHub profile > settings
(ii) From the left panel, click on Developer settings
(iii) Select Personal access tokens and click Generate new token (classic)
(iv) Pass the token name and select the expiration and scope
Your token would be generated just like shown below:
Now, copy the above-generated token to use in the further steps
Step 2: Form URL by collecting data about the file
After generating the token, the next step is to gather relevant details about the repository and the file that you want to explore. You need to form a URL in the below format:
https://api.github.com/repos/<owner>/<repo>/contents/<path>For example, if I want to read the “test.csv” file from my Repository(“https://github.com/Anshita-Bhasin/Cypress_Examples/blob/main/csv/test.csv”), then I need to initiate the process by accessing its corresponding URL.
For that, 3 key pieces of information are required: owner, repo, and path.
In the above example,
(i) the owner would be “Anshita-Bhasin”
(ii) repo would be “Cypress_Examples”
(iii) path would be “csv/test.csv”
Now, the URL formed would be as below :
https://api.github.com/repos/Anshita-Bhasin/Cypress_Examples/contents/csv/test.csvStep 3: Fetch data from GitHub Repository and write it to File in Cypress
Once you have the token and URL ready, now next step is to call the git API
Pre-Requisite:
- GitHub Repository
- Cypress installed on your system
Make a request to the git API by using the URL created in the previous step and passing the token as a header.
It’s always a good approach to execute the process manually first and then try to automate it. The screenshot below shows the data returned by hitting the git API (using Postman) with the URL.
The data returned by gitAPI includes various attributes like name, path, sha, type, content, etc. In order to use this data in the Cypress test, we need to decode the content data.
Now, let’s see how to automate the above flow.
Below is the automation code to fetch the data (csv file) from a private GitHub Repository in Cypress.
it('Fetch csv from github repo', () => {
cy.request({
method: "GET",
url: "https://api.github.com/repos/Anshita-Bhasin/Cypress_Examples/contents/csv/test.csv",
headers: {
Accept: "application/vnd.github+json",
Authorization: "Bearer <YOUR-TOKEN>",
},
}).then((sshresponse) => {
cy.writeFile(
"cypress/fixtures/csv/test-csv",
atob(sshresponse.body.content),
"utf8"
);
});
});Code Walkthrough:
The code mentioned above sends an HTTP request to a GitHub API to retrieve the contents of a CSV file and then saves it to a specified location using Cypress’s writeFile command.
The aim is to make a Get request to the URL, retrieve the content from the response body, and decode it as it is in an encoded format (as displayed in the last screenshot using POSTMAN).
Now, let’s try to understand the functionality of each step.
it
This is a test case definition using the Cypress test runner. The test case name is “Fetch csv from github repo”.
it('Fetch csv from github repo', () => {Request
cy.request() is a Cypress command that sends an HTTP request to the specified URL and the method: “GET”, specifies the HTTP method used to send the request.
cy.request({method: "GET"URL
url: “https://api.github.com/repos/<owner>/<repo>/contents/<path>”
This specifies the URL to which the request is sent. It is a URL of a GitHub API that returns the contents of the “csv/test.csv” in the “Cypress_Examples” repository owned by user “Anshita-Bhasin”.
url: "https://api.github.com/repos/Anshita-Bhasin/Cypress_Examples/contents/csv/test.csv"Headers
This specifies any additional headers to include in the request.
headers: {
Accept: "application/vnd.github+json",
Authorization: "Bearer <YOUR-TOKEN>",
},The below code specifies that the server should respond with JSON data in response.
Accept: "application/vnd.github+json"The below code specifies an authorization token to authenticate the request. In this case, it is a bearer token.
Authorization: "Bearer <YOUR-TOKEN>"Write to a File
then((sshresponse) => {
cy.writeFile(
"cypress/fixtures/csv/test-csv",
atob(sshresponse.body.content),
"utf8"
);
});.then((sshresponse)
This is a Cypress command that executes a callback function when the HTTP request completes successfully. The response object is passed to the callback function as a parameter.
cy.writeFile("cypress/fixtures/csv/test-csv",The above code will write data to a file at the path cypress/fixtures/csv/test-csv
atob(sshresponse.body.content),atob(sshresponse.body.content) => This is a Base64 decoder that decodes the contents of the “csv/test.csv” file returned in the response.
"utf8"“utf8” => This specifies the encoding(csv) of the file to be written.
Execution:
On executing the above code, csv file is saved into the Cypress project at the specified location.
NOTE: In this case, we are fetching CSV files from different GitHub Repository (other than your project’s repository)
Conclusion
In this tutorial, we learned how to fetch data from a CSV file hosted on GitHub using Cypress.
Ref Link — https://docs.github.com/en/rest/repos/contents?apiVersion=2022-11-28#get-repository-content
Git Repo — https://github.com/Anshita-Bhasin/Cypress_GitCSV-Fetcher/tree/main
Hope, you like the article.
Thanks for reading. Happy Learning! AB
