W3C HTML Validation for Twig Generated Pages
Pulsar, for those who don’t already know, is the open-source User Experience and Interface design system for the Jadu Continuum platform. Our company’s clientele is almost exclusively large scale institutions like Local Authorities, Higher Education and third-sector not-for-profit organisations. For that reason, high standards and quality are required for the code we write. We already test some of our code quality using linters for JavaScript and CSS, but we also wanted to check if the HTML structure of our design system components pass the validation of standards set by W3C.
Tools used: Grunt, Grunt W3C HTML Validation (NPM Package), Phantom JS, Casper JS
The problem we encountered originally was with our use of Twig as a templating engine. Because Twig generates our HTML using PHP there was no origin HTML file for the page loading in the browser, resulting in an absence of files to use as input to the W3C Validator. Secondary problems were the traversing of all the menu links, the proper naming of the output files and the error template presentation.
After a little bit of research, a solution was crafted. A harmonious combination of Casper JS, which is a navigation scripting and testing tool for Phantom JS, with good old regular expressions. Those dependencies were adapted and then integrated into our current task runner Grunt.
The first step is to add W3C Validator, Phantom JS and Casper JS dependencies to the project via NPM.
The second step is to create the casper.js file, preferably in the js directory of the project, to keep things tidy. There we can imitate basic user behaviour and visit all the links in the main menu. We now have a “map” of the website, an array of the visited links. The regular expression is trimming the unnecessary characters and produces an array of clean names. This array is used to name the output files on a second casper visit of the links, putting the HTML parts into separate and properly named files.
Those new HTML files are now saved in a dedicated folder within our tests directory.
The third step is to create the validation task and command within Grunt.
At this point we need to mention the need for an error template file which will help us present the errors in a nice readable way to start eliminating them.
We register the new task and with the command “grunt validate” we can finally execute the script.
The expected result is a list of errors and the lines in which file they can be found. A beautiful version of this can be found in /tests/validation/error_reports/ and looks like this:
At last we have a very good overview of the fixes needed!
To keep things tidy and avoid committing the error log related files, we suggest adding these to your .gitignore and/or .gitattributes files:
The application of the W3C Validator was originally implemented for the Pulsar design system but it was quickly adopted by the team in charge of one of our core products, Continuum CMS. In Pulsar we aim for a proactive approach with individual old and new components complying with the W3C rules. Especially, from the point we implemented the validator onwards we observe a natural scalability of quality code. This eventually leads to high-quality components that don’t need a second check when they are used within products.
In Continuum CMS the approach was slightly different because at the time nothing was yet validated. There the validator was used to better integrate an epic feature for front-end layouts design functionality. Of course, the end goal is to validate only the work done in Pulsar before allowing it to be used on the rest of the products. Doing that will significantly decrease testing times and ensure that our design system is providing high quality, valid HTML which we can automatically test when releasing new versions.
Feeling we have missed something or think parts of the code can be written in a more awesome way? Feel free to get in touch and contribute! The related work can be found in our open-source Github repo here.
