Pervasive code queries in the IDE

Elegant and accessible code queries with the Glamorous Toolkit

Search is a pervasive need in software development. Yet, the typical IDE rarely goes beyond a regular expression queries. We must do better than that.

One of the design principles behind Glamorous Toolkit is that search must be pervasive. It sounds grandiose, but once you setup to do it, it’s actually pretty straightforward.

Let’s start with an example: a search for all methods that are annotated with <gtView> and that reference the class GtMethodCoder.

How was this realized? On the left, we actually have a Playground with a code query:

#gtView gtPragmas & GtMethodCoder gtReferences

Executing the query opens an inspector on the result of the query which, in this case, is an instance of GtSearchIntersectionFilter. The filter instance knows how to present itself through a view through which the inspector diligently shows method widgets. Beside showing the method code, these method widgets also know how to match the input query to the abstract syntax tree (AST) nodes to produce the orange highlighting.

That’s it. That’s the whole tool. In fact, it’s not a tool. It’s an idea molded systematically into the environment.

But, wait. There is more. The query language is not a special language either. It’s plain Pharo. This means we can also take the result of the first query and refine it with another one:

self & #gtSourceFor: gtImplementors

Essentially, we have an explicit composition of filters. This approach makes the query “language” easily extensible. There already exists a number of predicates readily available.

The diagram below shows the class hierarchy for the current set of available filters. This is a visualization provided by Glamorous Toolkit live in the environment to help newcomers learn about the different filters they can use and how to extend the library with their own filters.

The SearchFilter class hierarchy in Glamorous Toolkit. The diagram is offered live by Glamorous Toolkit to help newcomers understand and extend the filtering logic. As a side note, the embedded editors are actually live and can be used in place to modify the code.

The concrete visualization shows the implementation of the ImplementorsFilter. The matches: hook method checks that the queried object is a method, and includesMethod: checks whether the selector of the queries method matches the filter.

Ok, but wait. All these classes look like small command wrappers around the existing model provided by Pharo. Why the aggravation? Why not just use the Pharo directly?

The main reason is to gain declarativeness and the tool support that can come with it. For example, note how when we search, the concrete appearance in the code is highlighted, and how this works even when we have a composite query.

And it is possible to combine a declarative query with Pharo code. For example something like this:

#gtExample gtPragmas select: [ :method | method package name beginsWith: 'GToolkit']

This would provide all the example methods (Oh, what are examples? Check this introductory article out) from packages starting with GToolkit. Such a query will not highlight the package name, but it has the advantage of offering quick prototyping.

Approached like this, search is not a feature. It’s a deep capability. Consequently, you should not judge it by measuring the directly supported use cases, but rather by the new kinds of scenarios that it enables.

Talking about scenarios, here is a short story.

In Pharo, there are a couple of special literals that are handled specially by the virtual machine. They are -1, 0, 1, 2, nil, true, and false. Not too long ago, we had to explore the impact of changing these. So, we queried for them.

It turns out that there are four places in the system that hardcode these values, and one of them does not do it as a literal array, but explicitly creates an array object. The query elegantly found and highlighted all of them.

So, in our environment, code search is available everywhere we can evaluate code. That is already something, but to make the story round we would need to complement it with the editing experience. And, indeed, the editors we see in the filter view are actually live editors!

With a simple twist, the Playground and the Inspector just became scoped code Editors. There are no tooling silos in a software environment.