Arthur Lee & Allen Kleiner | Pinterest engineers, Core Experience
The biggest bottleneck we identified was that eslint was running on a single thread. We used the worker-farm package to parallelize eslint runs across multiple cores. We then took the list of files that needed linting and distributed them to multiple worker processes. This gave us immediate gains with speedups of ~3x on an 8-core machine. This was reasonable given the overhead involved in starting worker processes in node.
This was a significant improvement, especially for our continuous integration (CI) jobs. However, we run eslint every time a file changes in our development environments so engineers have regular feedback about the state of their code. Running a CPU-consuming process every time a file changes would be too slow while also possibly slowing down more important processes, such as webpack. It became clear we needed a better solution for the dev environment.
Inspired by flow
We called the new eslint cli esprint (pronounced E-S-sprint). It uses a multi-threaded lint runner to lint files and spins up a background server to watch the source files using Facebook’s watchman. Whenever files change, watchman picks up the change, and esprint preemptively lints those changed files. Running the
esprint command simply queries the background server for the latest lint status. Since linting each additional file takes very little time, the runtime of esprint is in the order of seconds, with the exception of the initial run.
As a result, we have a fast linting solution that is suitable for both CI and development environments. Looking ahead, we plan to improve the reliability of the background server as well as performance using techniques like reusing eslint engine instances. You can find the code for the beta release on GitHub. We welcome any issues and pull requests!
Acknowledgements: Thank you to the Core Experience team and our web engineers for testing and giving feedback.