pySearch Dev Log: More Stories from a First Time Maintainer

pySearch is a command line tool I created when I first started learning Python to initiate web searches when I was using CLI and as pySearch started to pick up steam and contributions it also became the first project I acted as a maintainer for. Before maintaining pySearch it was hard to truly appreciate how much time and effort is required to maintain an active project.

When people first started contributing to pySearch it would be an understatement to say I underestimated the amount of time and effort it would take to maintain the project. It’s easy to say that a maintainer merges and mentors contributors on how to contribute but it’s hard to tell the amount of time and effort that involves until you’re on the other end.

Tests

In my previous blog posts on pySearch I mentioned how tests are important to avoid regressions as well as how I added a test framework to pySearch using pytest. While implementing tests to pySearch did help with reducing the amount of time it took to test code changes it did not change the amount of time it took to test new features.

One reoccurring problem that’s made testing pySearch features extremely difficult and time consuming is pySearch’s cross-platform support. pySearch is designed to run on all major OS and operate similarly, if not identically on different platforms. However this makes testing and troubleshooting the code rather hard as this means a maintainer needs to test the code at least twice because of the standout differences between Unix and Windows.

A standout situation where this was important was when browser selection was implemented. pySearch uses the webbrowser Python module to open the web browser after pySearch builds the link. The webbrowser module automatically registers web browsers it can detect so different browsers can be invoked as long as the module detects them. During testing we noticed that while Linux and MacOS registered web browsers relatively consistently this did not work correctly on Windows. After looking at the documentation and testing the behavior of the module we found that the browsers’ install directories had to be added to Windows environment variables to be detected by the webbrowser module. After this we decided that we would make a note of this in the README however this would not have been discovered if we had not tested on multiple OS.

Mentoring

Open source has this interesting dynamic where you get code from others and you give code in return .One of my most intriguing mentoring experiences occurred when I was mentoring a collaborator when they got stuck on implementing a ping to verify the validity of a search url. Originally I had suggested that they explore the ping subsystem call as it is implemented universally on both Unix and Windows systems. Unfortunately this did not work as ping only checks if the domain is up and does not verify the existence of resources and it doesn’t check if a domain simply redirects to a valid domain. For example: stackoverflow.ca would redirect to stackoverflow.com but if the browser tries to access a resource at stackoverflow.ca (eg. http://stackoverflow.ca/search?q=test) the URL will fail.

The collaborator was stumped and asked if I had any ideas. I knew that a curl call would work for Unix based OS but curl did not exist on Windows so I searched for a Powershell equivalent. After testing alternatives I found the Invoke-RestMethod call to be exactly what I was looking for and I reported my findings to the collaborator along with the example call that I tested. This experience was completely new to me as, until now I have never run into a situation where I had to test a fix that I wasn’t going to implement myself.