Out With The Old, In With The Tools
Dockstore recently released 1.12, heralding in a new definition of tools. In addition to simplifying what we call a tool here on Dockstore, we also streamlined tool registration by adding .dockstore.yml registration support. Other additions include better handling of Zenodo permissions and an overhaul of our documentation resources, so even if you do not register tools, you should find the process of using the Dockstore easier. All of these changes have the goal of making working with bioinformatics tools easier than ever before.
What Is a Tool Anyway?
First, let’s talk about tools. Common Workflow Language, better known as CWL, makes a distinction between tools and workflows, with the former being declared a `CommandLineTool` in its descriptor file while the latter is called a `Workflow`. In the world of CWL, a workflow can be a chain of CommandLineTools working in sequence. Additionally, a CommandLineTool (or the individual steps of a workflow) can but does not have to be containerized, usually via Docker.
Workflow Description Language (WDL) and Nextflow do not have a concept of tools — everything they do is simply called a workflow even if it only performs a single task. As with CWL, their individual tasks can but do not have to be containerized.
The Old Way of Registering Tools on Dockstore
Previously, to register a tool on Dockstore, we asked that a Dockerfile be provided. This was in service of our goal to encourage containization of tools. Containerization helps with reproducibility, and also saves a user time as they do not have to install prerequisites. But as some tools use Docker images that are not maintained by the same author of the tool itself, we found that requiring a Dockerfile for tool registration was not acceptable for all of our users. It also created confusion involving versioning, as the tool was considered by Dockstore to have a new version when a new Docker tag was generated, not when the descriptor file was updated, nor when the tool had a versioned release on GitHub (if that was where the tool was hosted).
Tools on Dockstore in 1.12 and Beyond
Starting in 1.12, we are now handling tools more like how we handle workflows. Versioning is now based on branches, tags, and releases from GitHub. We also no longer require that a Dockerfile be included when registering a tool, although we still encourage referencing a publicly available image inside a tool’s descriptor file to maximize reproducibility, and to have that image based upon a well-known library image or one with a Dockerfile available, such as Docker Hub’s official images or the BioContainers image library. These changes are all in line with how Dockstore already treats workflows, and should make things simpler, especially for our CWL users. Most importantly, we added support for the registration of tools using the Dockstore GitHub App. This allows tools to be registered in a way that automatically synchronizes your Dockstore entry with GitHub. Every new branch or release that has a .dockstore.yml file on it will appear on the Dockstore page for that entry.
We talked about the benefits and overall process of synchronization via .dockstore.yml files and the Dockstore GitHub App in a previous blog post. Check it out for more information on the process.
What Else Is New in 1.12?
Documentation Improvements
As mentioned above, we have added many example and template .dockstore.yml files to make registration via the Dockstore GitHub App even easier. Tools and workflows each get their own page which includes a basic mostly-blank template, several filled-out examples, and one annotated template with comments explaining every possible field.
Our tutorials for “Getting Started with CWL” and “Getting Started with WDL” have been modified to better fit how we now categorize tools. We also updated some of our older documentation to highlight our newer features. Additionally, to make it easier to find what you’re looking for quickly, we improved our landing page and provided an index page for docs relating to the Dockstore GitHub App.
UI Improvements
Science is collaborative, and it is not uncommon for a workflow or tool to have multiple authors. We now support showing multiple authors for an entry, adding authors via the .dockstore.yml, and automatically keeping this data up-to-date via ORCID. But that’s not all — it is now easier to add labels to your entries to make them easier to find by other users. These labels are used by Dockstore’s search engine to help users find the perfect tool or workflow for their scientific analysis.
Website Titles
If you currently have 20 tabs open, this is for you: Previously, when on a public Dockstore entry, the title of the webpage as it appears on a tab would be “Dockstore | Workflow” or “Dockstore | Tool”. Not only did this sometimes make switching between tabs difficult, but it also affected your ability to find a workflow or tool in your browser’s history, as some browsers show only the title of a webpage in their history. Now, the name of the tool or workflow will be displayed in the title.
Multiple Search Improvements
We made searching easier by preventing a search from being executed while you are still typing. Now, a search will only happen when you press the enter key. Topic sentences now show up in search alongside titles and authors. We also fixed a few bugs with the searching process.
What’s Next?
Dockstore has been doing a lot of work behind the scenes on security and compliance requirements for our integration with NIH systems. These are of course ongoing tasks that never truly have a finite stopping point, but with the infrastructure we have built, we now have more resources to dedicate to other areas of development. We have further plans to streamline working with Dockstore, both for users and ourselves — expect to see more frequent updates as we change how we do releases.
