Don’t be afraid of headless Chrome! Learn why and how to use it for Ember testing

Jen Weber
Jen Weber
Sep 5, 2017 · 6 min read

Last updated 9/1/17, Ember CLI 2.15. Special thanks to Scott Newcomer and Ben Demboski for helping me to debug, Karl Becker for editing, and Tobias Bieniek for some CLI tips!

After a few hours debugging my EmberJS testing suite that refused to run from the command line, I ditched PhantomJS and switched to Headless Chrome. Let’s break down what it all means, how I did it, and what the impacts are.

What even is Headless Chrome?

It’s a way to run the Chrome browser in a headless environment. Essentially, running Chrome without chrome!

Image for post
Image for post
(Alt: what is this, I don’t even….)

Right. Here’s a better definition from Wikipedia:

A headless browser is a web browser without a graphical user interface. Headless browsers provide automated control of a web page in an environment similar to popular web browsers, but are executed via a command-line interface or using network communication.

Starting with Ember CLI 2.15, headless Chrome is the default for testing in EmberJS. If you’re working with an older Ember app, I have good news - you don’t need to upgrade your app in order to try out headless Chrome. As a matter of fact, you can use the latest CLI release with almost any older Ember app version.

Why is a “headless” environment needed?

So, why not just run the tests directly in a normal Chrome browser? If you have an Ember app, run ember serve, and visit http://localhost:4200/tests, you can actually watch your tests run in real time, or pause them and look at the app state visually. However, where headless browsers really shine is when they are used for Continuous Integration testing, commonly referred to as CI. In production apps, it is common to use a service that automatically runs your test suite as code is committed. And for the most part, those tests run on a server, not in “normal” browsers. For example, check out the open Pull Requests for part of the Ember website, especially any with a red x next to them. Whenever a pull request gets opened on GitHub, you can see whether it passes the tests.

Image for post
Image for post
This is not my pull request. I swear. (Alt: image showing multiple test failures on a GitHub pull request)

Why not use PhantomJS?

Headless Chrome is coming. I think people will switch to it, eventually. Chrome is faster and more stable than PhantomJS. And it doesn’t eat memory like crazy. I don’t see any future in developing PhantomJS.

As the maintainer noted, PhantomJS has some problems. I had one of my own: none of my tests would run. Everything was fine in a fresh app, but some unknown part of my real app wasn’t compatible after introducing some well-established dependencies. I ran ember test, but before any tests had even begun, I was greeted with these errors:

not ok 1 PhantomJS 2.1 — Global error: SyntaxError: Unexpected token ‘}’ at http://localhost:7357/assets/vendor.js, line 120177not ok 2 PhantomJS 2.1 — Global error: Error: Could not find module ember-metal required by: ember-testing/support at http://localhost:7357/assets/test-support.js, line 58not ok 3 PhantomJS 2.1 — Global error: ReferenceError: Can’t find variable: define at http://localhost:7357/assets/ember-bio-bright.js, line 5not ok 4 PhantomJS 2.1 — Global error: ReferenceError: Can’t find variable: define at http://localhost:7357/assets/tests.js, line 3not ok 5 PhantomJS 2.1 — Global error: ReferenceError: Can’t find variable: EmberENV at http://localhost:7357/4215/tests/index.html?hidepassed, line 38

I threw everything at this error. Blowing up node modules, removing all but the simplest test, reinstalling the EmberCLI, installing/uninstalling PhantomJS, digging into the vendor bundle, sharing angry cat GIFs, lighting some incense… nothing.

After a little bit of Q&A with a couple other developers, it was suggested that I try out Headless Chrome to see if the errors became easier to debug.

The errors did not become easier to debug.

The errors simply disappeared.

How to make the switch

What is Testem?

Image for post
Image for post
See this? That’s Testem in action, resulting from `ember test — server`. As you can see, my latest work on ember-api-docs is going GREAT. Everything will be fine and I definitely know what I’m doing. Not. (Alt: terminal window where the testem runner shows 175 test failures)

What negative impacts could Headless Chrome have?

It’s also new-ish. It was shipped with Chrome 59, but developers have had ways to use Chrome before that. The big question is, if you’re already doing CI testing, how well does your vendor support it? A lot of the big players have been quick to implement it, but it’s possible you might run into some issues.

Finally, you might have some tests that pass in PhantomJS and fail in Headless Chrome, which means it’s time to do some debugging. Of course, I experienced the reverse, where all my problems magically went away when I switched headless browsers, but problems are not uncommon. This is a pain point akin to things looking great in Chrome but they explode in Firefox. For this reason and to make their tests more robust, some organizations run their tests in multiple headless tools.

When to switch

Happy Headless Testing!

Image for post
Image for post
Artwork by IrenHorrors, shared under Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License

Frontend Weekly

It's really hard to keep up with all the front-end…

Jen Weber

Written by

Jen Weber

Builds Web 3.0 at Cardstack. Ember.js core team. Science nerd. www.jenweber.me

Frontend Weekly

It's really hard to keep up with all the front-end development news out there. Let us help you. We hand-pick interesting articles related to front-end development. You can also subscribe to our weekly newsletter at http://frontendweekly.co

Jen Weber

Written by

Jen Weber

Builds Web 3.0 at Cardstack. Ember.js core team. Science nerd. www.jenweber.me

Frontend Weekly

It's really hard to keep up with all the front-end development news out there. Let us help you. We hand-pick interesting articles related to front-end development. You can also subscribe to our weekly newsletter at http://frontendweekly.co

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store