A means to an end-to-end

Automatically re-run your test suite with Protractor

Because I had some spare time this week to try to get our test suite up to scratch, I started thinking about how we could encourage ourselves to write tests earlier in the process. One of our issues was that tests were added (if they were at all) only after developing and QA-ing features and bugs. This is very tempting because developing with hot reloading is so much faster and easier to debug than running your test suite every time you change a file. But we should know and do better. I figured I should try to find a way to combine the development speed of hot reloading with the practice of writing tests before and while developing the actual feature. (Note: while I’m using Webpack, it should be rather trivial to use this concept with other bundlers).

Re-running your test suite when a file changes is not that hard — watch your test files, hook in to a change event from your bundler, and re-run protractor. The only issue with this is that it’s relatively slow: protractor needs to restart and close the browser between tests, do some bootstrapping stuff, login, etc. This isn’t so much of an issue when you’re running the entire test suite, but if you just want to re-run one test, it quickly becomes a nuisance (and easy to abandon entirely). If we want it fast, we want to avoid these steps: the browser has to stay open, the user has to stay logged in, et cetera.

Getting the building blocks

Couple things we need here from Protractor: the ConfigParser to parse the protractor config file, the Runner to set up the testing environment, and a Jasmine adapter which runs the tests for us. We also need a watcher, in this case we just use Webpack’s watch library watchpack. (I also included Plugins, which is Protractor’s plugin API, but I’m not sure we actually need it.)

Creating the test runner

We need a test runner which a) opens the browser, and b) runs the given tests at any time, c) keeps the browser open. There are three gotchas:

• Jasmine uses require() to include specs (via describe()). Because of require’s cache, this means that once a test has been included, it cannot be included again. To mitigate this issue, we delete the included specs from require’s cache.
• jasminewd2 is a small library which modifies jasmine’s methods to work more seamlessly with webdriver’s control flow (think promises). However, that doesn’t happen when the test is run for the second time, and the tests fail, because jasmine no longer waits until the promise is resolved. Again we have to delete jasminewd2 from the cache to ensure jasmine’s methods are able to deal with promises.
• Once the tests have completed, the process closes. To keep the process open, and ensure the browser doesn’t close, we tell the browser to wait until the next test starts.

Running the tests when the files change

Lastly, we have to re-run the test suite when a test file changes, or when the bundle updates. When a test file changes, we store the path and run the tests. When the bundle changes, we run the most recently changed test. And there it is: super-fast, super-easy reloading, which will hopefully make it easier for us to write our tests first.