On-demand environments for running end-to-end tests

Sreenadh T C
Jul 31, 2019 · 4 min read

Be more confident about your code changes by adding end-to-end tests that run for each deployment you create on Dockup.

End-to-end testing is a technique used to verify the correctness of an application’s behavior when it works in integration with all its dependencies.
Running end-to-end tests have become exceedingly complicated over time as companies embrace service oriented architecture and monoliths turn into micro-services.

In this blog post, we’ll see how to use Dockup to automatically spin up on-demand environments to run end-to-end tests for every pull request.

We will be explaining this based on Cypress.io but you can follow the similar steps for configuring your favorite E2E tool to run alongside Dockup deployments.

For ease of understanding, let’s use a simple VueJS app that implements a TodoMVC.

We will keep this source under a common project folder, say todomvc-app/ and also create another folder, say todomvc-app/e2e/. We will write our tests inside this. Cypress test specs are kept under a sub-directory called "cypress" and we will have a cypress.json file inside our e2e folder. See more on how Cypress tests are written from their docs.

Once you have your test specs ready, we need to add a Dockerfile and the whole directory structure would look something like this:

todomvc-app/
|
|---- src/
| |---- index.html
| |---- app.js
|
|---- e2e/
| |----cypress/
| | |---- fixtures/
| | |---- integration/
| | | |---- add_todo_spec.js
| | | |---- mark_todo_spec.js
| | |---- plugins/
| | |---- support/
| |
| |---- cypress.json
| |---- Dockerfile
|
|---- package.json
|---- Dockerfile

Since the test cases are to be run for several deployments, we will be keeping the baseUrl config value for cypress as an initial dummy URL, and then we will override it with env variables. This is documented by Cypress here.

Container for the actual todo-app

Assuming that you have already added the container for the actual app while creating a Dockup Blueprint for your todomvc-app (as shown in figure above), we will add a new container holding the image source details. If you are new to Dockup Blueprint, head over here to read more about creating one.

Take care about the Dokcerfile path here, as this is the one which resides inside our e2e folder.

We will also have to add CYPRESS_BASE_URL env for Cypress to receive a public endpoint for the deployment. This can be done using the Environment Variables Substitution feature ( Refer DOCKUP_PORT_ENDPOINT_ ) in Dockup.

The cypress container would exit with the overall number of error the test had.

Container form for e2e

That is all you need to do to have a working cypress end-to-end test running alongside each of the deployments.

Since containers inside a Dockup deployment spin up when they are ready and not sequentially, you will need a shell script that waits for the UI endpoint to be live before you start to run tests. The script can simply fail when the endpoint is not live, upon which the Dockup container would restart.

#!/bin/shset -x
set -e
echo "Checking if the endpoint for testing is ready..."response=$(curl --write-out %{http_code} --silent --output /dev/null "$CYPRESS_BASE_URL")
if [[ $response != 200 ]]; then
# exit the script
exit 1
fi

Cypress has its own docker image configured to run on several CI tools, which you can use it on Dockup as well, without much changes. All you have to do is, put the cypress/folder in the same level as the Dockerfile as their images look for it in the root directory, and as soon as the containers spin up, cypress run command would run. It is however not recommended to do this on Dockup due to the reason mentioned above. Instead, have the script take care of running the cypress command when the endpoint is up.

Now you can go ahead and deploy this blueprint and have it run the E2E tests for you. Your containers should spin up and the e2e tests should start running. While you wait for them to complete, you can also take a look at the logs.

Image builds are ready

A successful deployment with your e2e tests passed would look something like this.

E2E test has passed, and hence the container has a success check

Checks also send updates to GitHub if the deployments are triggered by PRs.

An example of how Dockup sends updates to GitHub PRs

In the case of cypress, it would exit out with a non-zero exit code when there are failures, and thus the container would also fail, suggesting there are failed test cases.

Not using Dockup already? Click here to start for free.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store