Cypress vs. Testcafé
All you need to know
What’s the difference between Cypress and Testcafé? Well put in a nutshell: Imagine Cypress would have a full-feature paid version and a free version with just the basic features… then Testcafé is like the free starter version of Cypress.
1. But let’s see what you actually need to pay for and what you get for free.
Testcafé is available in an open-source and a paid version called “Testcafé Studio”. The main extra feature of the studio version is the so called “visual test recording” and advanced cross device testing. Visual test recording is supposed to autogenerate your test code just by clicking on the elements in the UI. Yes! Autogenerated test code… sounds scary, right?
Cypress on the other hand is completely open-source. Only if you want to use the “Cypress Dashboard” with more than three users you’ll get charged, too. Do you need the Cypress Dashboard? Well, not really. The dashboard can help you to make the test results visible and access screenshots and recordings without any Jenkins hustle. To activate it you’ll need to add a generated key in your configuration file and that’s it. Cypress will now upload the test results of each test run on its own server and present them glammed up to you. For a lot of projects this will even cause compliance problems. So just invest some more time in your Jenkins setup and you’ll never be in need of the “Cypress Dashboard”.
2. With which tools does the test framework support me while writing tests?
Here Cypress is the undisputed king of the jungle. Its developer support is just amazing! The “cypress open” command will open a little browser window for you with an overview of all existing tests. A click on the test will start the execution in a new window. You’ll see your application on the right and a history of all test actions on the left. After the test run finished you can inspect each action with the respective application DOM snapshot on the right. You can go back and forth in time and look at each action as often as you want to. If you’re not sure, whether a selector really queried the right element, you can click on the action in the cypress history and it will print the DOM element it found to the browser console. Debugging like a charm!
But Cypress will also support you while writing your tests. Most noticeable feature is that it runs in “watch mode”. You might know this feature from other testing frameworks like Jest. It means that a change in your codebase will automatically trigger a new test execution in Cypress. No more cumbersome re- and re- and re- and re-starting of tests! If your machine handles it you could also let Cypress constantly check your changes while working on production code. It will save you all manually UI clicking while developing new features from the very beginning. That’s test driven development on a new level!
Additionally Cypress offers you a feature called “Selector Playground”. If you enable it you can click on your UI elements and Cypress will display an appropriate selector for this element. Honestly I was very thrilled about this feature in the beginning, but it turned-out to be less helpful than expected. Usually I only use it when I know for sure that some element already has a data-testId but I forgot the name.
And Testcafé? Well you can pay and get awesome autogenerated test code… okay, just kidding. Testcafé supports you with a breakpoint feature. You can pause the test execution and continue action by action but only forward. For debugging you’ll need to use the tools your browser offers you. That’s all it has? Äh… let me think. Oh you can also run the tests at half-speed. Very useful feature to show your tests to your boss. But besides that nothing which will increase your developer productivity.
3. And how about Browser compatibility?
That is currently the only weakness of Cypress compared to Testcafé. Both offer support for Chrome, Edge and Firefox, but only Testcafé will allow you to run automated tests on a Safari browser. If Cypress will support testing with Safari one day is still unclear. Currently there is no active development taking place in order to get it running (see here).
Now the big question is: Should that stop you from using Cypress, if your customer asks for Safari support? Well, not really. In order to run your tests against a Safari browser you’ll need a Macintosh operating system. Meaning, you can of course run your fancy Testcafé tests against a Safari browser on your local MacBook. But I’ve never heard of anybody who did setup a Macintosh Server with Safari installed for automated end-to-end testing in a continuous integration environment. If you’re really serious with cross-browser testing you can indeed use a workaround. There are cloud services like browserstack.com which will let you run your automated tests against any possible device or operating system. The extra charge will most likely convince your customer that automated Safari testing might not be as required at all.
So summed up, even if your test framework supports Safari, you won’t have a big advantage of it.
4. Could there be any other reason why you run into problems when you use Cypress?
There is one controversial issue regarding cross-domain testing regarding cypress. If your application directs the user to another domain during the tested workflow, Cypress won’t allow you to test the content of the external homepage nor interact with the displayed elements. This could potentially mean that you need to adjust your login or payment test. While Testcafé will let you continue to interact with foreign functionalities, Cypress will force you to mock those external calls. For example instead of testing the real third-party login you can generate the required token yourself and add it to the header.
Which kind of test workflow is better, is of course a question up for debate. Testcafé allows you to naturally imitate the user’s workflow, while Cypress only tests code which is under your control and responsibility — resulting in (hopefully) more stable tests.
Conclusion: In my humble opinion there is no more reason to use Testcafé. And I even don’t think it has a future at all, if they don’t show up with an equally good debugging support.
Passion, friendship, honesty, curiosity. If this appeals to you, Comsysto may well be your future. Apply now to join us!
This blog post is published by Comsysto Reply GmbH
