A Solar Eclipse and the Story of Adding a Lunr.js-powered Search to Code.gov
I started the week of the solar eclipse by taking a day off and traveling to Greenville, SC to view the total solar eclipse. The total eclipse gave me a view of the sun that was so beautiful and outside the norm — it’s an unforgettable experience and I would recommend to anyone to experience it yourself the next chance you get.
After the drive back to Washington, D.C., I started the work week with a strangely related mission for Code.gov, a platform we are designing to facilitate the reuse of code across the federal government and sharing of federally funded open source code. We’ve wanted to add search to our site for some time now. Without having an Authority to Operate (ATO) the search engine that would power it, though, we’ve delayed launching the feature. For those outside the government, you can think of an ATO as a rigorous check to validate the security of any app released to the public by an agency.
The week before the eclipse, I had an idea that would allow for us to get search into our user’s hands before the ATO completed. We could add browser-based searching that would give our users a little bit of the power they would get if we had a server-side search engine. This solution was an open source library called Lunr.js — a play on the name of a popular back-end search engine, Solr.
Adding the library to the project was rather simple. Within a day, I had a prototype of a typeahead search for the federal open source repositories catalogued on the site. By the end of the week, I enhanced the typeahead search to include agencies and built out a search results page. What follows is a simple guide to get Lunr.js going in an Angular 2+ project.
First, I issued the command to add the dependency:
$ npm install --save lunrThen, I created a service that would maintain the state of this index. Below is how setup of the index looks:
Above we import lunr as well as repos, which is an Array of repositories in a JSON file that we use to populate the site. One of the things I chose to do was map our refs to the entities they refer, which makes the next step of returning search results much easier. Lunr’s API documentation is straightforward, and I referred to it often while building out the service.
Next, we define a method within our class for searching this index. Below is a simplification of it:
Since we want to provide typeahead functionality, we look for all words that are the current query (say “ag”), as well as ones that start with that query (“agriculture”, “agency”, or “agile”).
The next line uses the Lunr index to search for our query, and then converts those results into our repositories.
Finally, we use an RxJS Subject to communicate the change in state to any listeners.
That was essentially it for the logic required to allow our visitors to search for repositories. Lunr.js doesn’t have the full feature set of a robust, server-side search engine, but it is extremely valuable in the use case where you want to allow users to search your content but have a limited back end. With the rise of static sites, Lunr’s potential applications are more widespread than you’d initially think.
Wrapping up—whenever I’m given a new challenge, I can’t thank the people who write open source software enough for making my life as a developer much, much easier. The hard work was done by someone else (in this case, Oliver Nightingale, or @olivernn on GitHub), and we all benefit for it. Without the library, this feature would have taken months instead of a week. Reasons like this are why I’m excited for the mission of Code.gov to promote and expand open source in the US government. If open source interests you, follow us on our path to building the federal platform for open source advocacy!
