Failing CI build when React Warnings are raised

Unit tests, style checkers and other static analysis tools are really useful but many times they are not enough to catch bugs before they go live. Code reviews are also great but sometimes it's not enough either.

At Crave we are using a test setup with Mocha + jsdom and a fairly strict config for JSLint. We are also making sure that all pull requests are revised by one person different than the original author of the code. Even with this setup sometimes we encounter issues in our code. When such cases happen there is a questions that's we try to raise: "How this could we automatically detect this issue before it happens again?".

Last week we were caught by that situation during a small/medium refactor. Basically what caused the issue was that we have changed the API of a component that was used in many different places but we didn't update all places using the old component. As run time React did log warnings in the browser (using console.error) but our tests wouldn't catch it.

The first solution that came to our minds was to write "better unit tests" for that component. Obviously that would work, but we would be relying on more code which implies in more reviews and potentially more problems.
Since React provides us Prop validation using PropTypes it seemed logical that PropTypes validations would be an interesting way to go.

After some research I've found Bonnie Eisenman text about PropTypes validation. She suggested to write tests using mocks to spy on console errors and than fail the tests if needed. Christopher Cook also suggested the implementation of a custom Jasmine matcher to write more legible tests.

This turned to be a very useful idea but since we are not using Jasmine we would have to adapt it. Also Bonnie's idea would require to write a test for each component.

After some more experimentation we though about using the Mocha's beforeEach and afterEach hooks. Using those hooks we would be able to check the console log for each test and also cause the test to fail if needed.

That almost worked but since we were already mocking console.error using SinonJS the tests started to fail because of the "double patching". Since we didn't want to make those tests rely on a global patching we went for a different alternative: monkey patching.

Basically we used M0cha's setup file (setup.js) to replace our console.error by a function that would perform the same task but would throw an error if it's called using some of React's Warning messages.

That's how it looks in the end:

With this approach if a component tries to use another component passing properties that doesn't exist the test will fail and the error message will be raised. That approach is currently working for React 0.14.x but we haven't tested with React 15 yet (there is a chance we'll need to update the ERROR_MESSAGES constant).

If you have any suggestions please let us know in the comments!