Evaluating @Deprecated classes in Java systems with Glamorous Toolkit
An example of molding custom source code analyses
Glamorous Toolkit, the moldable development environment, ships with extensive software analysis support for various languages. These analyses can be combined and integrated with the rest of the environment to assist developers in assessing their own system.
In this article we exemplify how this works through a classic example: evaluating @Deprecated classes in Java systems. We pick this problem because it is one of the simplest evolutionary problems that is often taught in introductory courses. Yet, in our case, we show how to address it without reading code, merely by constructing custom interactive analyses.
The software modeling part is based on FAMIX, a family of meta-models for representing various facts about software systems. FAMIX is developed within the context of the Moose analysis platform. Glamorous Toolkit integrates FAMIX and the core of Moose into the overall development environment.
On deprecated classes
Marking a class as deprecated indicates that the class is no longer considered important and it might cease to exist in the future. This can help in many cases, for example when migrating APIs.
Let’s imagine we have a system with multiple classes being annotated as deprecated. The first question is: “which of these can we remove?”. Answering this question essentially boils down to finding the classes annotated with @Deprecated and then selecting those that are not used anywhere (we ignore reflection for this exercise). Once we address the first question, the follow up one is: “what do we do with the classes that we cannot remove right now?”. The usual answer to this one is: “it depends”.
In any case, both of these problems require analysis. We can approach them manually and click around the IDE. But, that’s not what you are here for. So, let’s explore them differently.
Setup
As an initial case study we use ArgoUML, an open-source Java applications for creating UML diagrams. We use version 0.34 of this application, which has around 2400 classes and ~200k lines of Java code. After we finished creating the analysis, we also apply it to a second project, namely Apache Lucene Solr.
To begin our analysis we need to load a model of ArgoUML into Glamorous Toolkit. We achieve this by means of an MSE file that must be exported by an external exporter. If you are reading this tutorial in Glamorous Toolkit, you can download the source code of ArgoUML, and an already created MSE model file using the snippet below. The MSE file was created using jdt2famix.
targetFolder := 'models' asFileReference ensureCreateDirectory.
archiveFileName := 'ArgoUML-0-34.zip'.
archiveUrl := 'https://dl.feenk.com/moose-tutorial/argouml/'.ZnClient new
url: archiveUrl, archiveFileName;
signalProgress: true;
downloadTo: targetFolder.
(ZipArchive new
readFrom: targetFolder / archiveFileName)
extractAllTo: targetFolder.targetFolder
The snippet creates a new models folder in the Glamorous Toolkit directory, and downloads and extracts an already prepared archive for this tutorial that contains the code and the MSE model file (the archive is about 33 MB). We can inspect the models folder to check if the download went ok.
We can load that model by executing another code snippet.
modelFile := 'models' asFileReference
/ 'ArgoUML-0-34'
/ 'ArgoUML-0-34.mse'.
model := MooseModel new
importMSEFromFile: modelFile.
modelNow we have a loaded model of our system that we can inspect and explore.
Alternatively, if you want to also build the model file locally from source code, follow the steps from Appendix Section.
Locating deprecated classes
When first inspecting a model we see various groups of information about the analysed system, like classes, methods, annotations, comments, etc,. We can select any of those groups to further explore. As our analysis is related to classes we can select All model classes.
By default we see the list of all classes from the system. At this point we are only interested in deprecated classes. We can quickly select only those; drag the horizontal bar at the bottom of the list to bring into view a playground and enter:
self select: [ :each | each isAnnotatedWith: 'Deprecated' ]Press the Inspect button (or use a shortcut: Ctrl+g on Windows and Linux, and Cmd+g on Mac). This results in a new pane that spawns to the right, containing 25 classes marked as deprecated.
Just to make sure that we indeed got deprecated classes, let us investigate one of them and select to view the source code.
Let’s step back for a moment. We see 2 panes, but our inspection session consists of 5 panes. The first pane is the playground. The remaining four represent objects and the inspector offers multiple views on those objects. Furthermore, each pane is also represented by a rectangle in the scroll bar from the top. To adjust the amount of visible panes, we can resize them.
Another way to search for deprecated classes instead of using the playground attached to the group of classes, is to add a new snippet in the initial playground that gets the list of model classes from the model and selects the deprecated ones.
deprecatedClasses := model allModelClasses select: [:each |
each isAnnotatedWith: 'Deprecated' ].This snippet also defines a local variable, deprecatedClasses to store the result. The method allModelClasses returns the group of model classes from the model. We will continue to use this way of defining the analysis, as at the end our entire analysis will simply consists in several snippets of code.
Going back to our original problem, we now have the 25 classes that are deprecated in our system. Next, we need to check which ones of those are not used. Or if we think in terms of clients and providers, which of the deprecated classes do not have client classes. We can add another snippet in the playground that further filters the list stored in deprecatedClasses .
deprecatedClasses select: [ :each |
each clientTypes isEmpty ].
Out first part of the analysis is done. We get 14 classes that we could just remove, as they are not used within the system. This leaves us with 11 classes that cannot be removed because they are still being used. So, what should we do about these?
stillUsedClasses := deprecatedClasses select: [ :each |
each clientTypes isNotEmpty ].
Visualizing deprecated classes
Now that we have the list of deprecated classes from the system, it would be great to know how these classes are being used. Perhaps a deprecated class is used in several places. Or maybe we have one non-deprecated class that uses multiple deprecated ones. Or it can also be that multiple deprecated classes call each other. We could try to create lists and tables, but they would not be quite ideal to exhibit these patterns. So, let’s build a visualization instead.
Add another snippet in the playground and execute:
view := GtMondrian new.
view nodes
with: (stillUsedClasses,
(stillUsedClasses flatCollect: #clientTypes)) asSet.
view layout grid.
viewThis shows us a simple visualization containing the 11 deprecated classes and their client types.
What we get is an interactive picture. Clicking on a node reveals the details of the corresponding class to the right.
It’s not quite convenient that all classes look the same. Let’s distinguish between the deprecated and the non-deprecated classes:
view := GtMondrian new.
view nodes
shape: [ :each |
BlElement new
size: 5@5;
geometry: BlCircle new;
background: ((each isAnnotatedWith: 'Deprecated')
ifTrue: [ Color red ]
ifFalse: [ Color gray ]) ];
with: (stillUsedClasses,
(stillUsedClasses flatCollect: #clientTypes)) asSet.
view layout grid.
view
Ok, now we see the classes, but what are the dependencies?
view := GtMondrian new.
view nodes
shape: [ :each |
BlElement new
size: 5@5;
geometry: BlCircle new;
background: ((each isAnnotatedWith: 'Deprecated')
ifTrue: [ Color red ]
ifFalse: [ Color gray ]) ];
with: (stillUsedClasses,
(stillUsedClasses flatCollect: #clientTypes)) asSet.
view edges connectFromAll: #clientTypes.
view layout grid.
view
Ok. Perhaps if we arrange the graph a bit better…
view := GtMondrian new.
view nodes
shape: [ :each |
BlElement new
size: 5@5;
geometry: BlCircle new;
background: ((each isAnnotatedWith: 'Deprecated')
ifTrue: [ Color red ]
ifFalse: [ Color gray ]) ];
with: (stillUsedClasses,
(stillUsedClasses flatCollect: #clientTypes)) asSet.
view edges connectFromAll: #clientTypes.
view layout force.
view
Ok. Much better.
We can distinguish several distinct situations. Two deprecated classes (in red) call each other and they can be simply removed. There is one non-deprecated class using two deprecated classes. And there are 3 deprecated classes that are being used by several other non-deprecated ones. This picture offers us the opportunity to choose our path by taking more details into account at the same time. For example, an interesting candidate to investigate is the one class that uses two deprecated ones.
System-specific customization
Until now the view that we created is generic and can be applied on any Java system. However, we can often gain more insights by taking into account information specific to the system that we are exploring.
In particular, the view provides an insight into which class might be more or less expensive to remove. However, to evaluate what to do next, we’d need an idea of value, too.
The ArgoUML project is separated into to main components: model and user interface (UI). Hence, when looking at the deprecation analysis it can be useful to know if classes belong to the UI or not. To show this we can create a specific view that colors the border of UI classes in blue. To identify UI classes we rely on the fact that the package name contains the ui component in its name.
view := GtMondrian new.
view nodes
shape: [ :each |
BlElement new
size: 5@5;
geometry: BlCircle new;
background: ((each isAnnotatedWith: 'Deprecated')
ifTrue: [ Color red ]
ifFalse: [ Color gray ]);
border: (BlBorder paint:
((each mooseName includesSubstring: '::ui')
ifTrue: [ Color blue ]
ifFalse: [ Color transparent ])) ];
with: (stillUsedClasses,
(stillUsedClasses flatCollect: #clientTypes)) asSet.
view edges connectFromAll: #clientTypes.
view layout force.
view
This very small customisation to our view gives us precious insight. We observe that most deprecations are related to UI classes. Also clients of deprecated UI classes are only other UI classes.
In this situation, which is more important to refactor first: the UI or the model? It depends! Of course, it depends on the context of the system. We cannot provide a value judgement from outside (ok, sometimes we can, but very rarely). Still, now we have a more detailed tool to guide the conversation.
Exploring different views
We can already use the created view to reason about deprecated classes and take decisions on how to approach them. However, the granularity of the view is still at the class level, and it does not given us much information about the actual classes:
- Are they small or large?
- Do they have many methods?
- Are many of their methods used?
It could be that only a few methods from those deprecated classes are really used. Or that they contain many used methods. Next we will adapt our view to try and address these questions.
As we add more details, the code to create the view gets larger. And we need to have more knowledge about the graphical framework. Nonetheless, the effort to build such kind of views can still be worthwhile as it reduces the amount of code we need to read in order to make relevant decisions about our software systems.
Also in the views that follow it is not needed for the read to closely follow the code of the view. The main idea that we try to convey is that analyses can be as detailed as we need them to be, if that helps us reason about our systems.
How big are deprecated classes?
Until now we did not take the size of classes into account. We can fix that by setting the size of a class based on its number of methods.
view := GtMondrian new.
view nodes
shape: [ :each |
BlElement new
size: (each numberOfMethods min: 50) asPoint;
geometry: BlCircle new;
background: ((each isAnnotatedWith: 'Deprecated')
ifTrue: [ Color red ]
ifFalse: [ Color gray ]);
border: (BlBorder paint:
((each mooseName includesSubstring: '::ui')
ifTrue: [ Color blue ]
ifFalse: [ Color transparent ])) ];
with: (stillUsedClasses,
(stillUsedClasses flatCollect: #clientTypes)) asSet.
view edges connectFromAll: #clientTypes.
view layout force
strength: 0.15;
charge: -250;
length: 70.
view
What deprecated methods are actually used?
We can see now that some deprecated classes are indeed quite large and others very small. But how much code in those classes is actually used?
To answer this question let’s show for each deprecated class the methods it contains that are actually called from other classes, and for client classes the methods that call deprecated methods. For this view we no longer color the border to not have that many details.
deprecateMethods := (stillUsedClasses flatCollect: #methods)
select: [ :aMethod | aMethod clientTypes notEmpty ].
clientMethods := (deprecateMethods collect: [ :aMethod |
aMethod clientMethods reject: [ :each |
each parentType = aMethod parentType ] ]) flatten.
interestingMethods := deprecateMethods, clientMethods.view := GtMondrian new.
view nodes
shape: [ :each |
BlElement new
background: ((each isAnnotatedWith: 'Deprecated')
ifTrue: [ Color lightRed ]
ifFalse: [ Color veryLightGray ]) ];
with: (stillUsedClasses,
(stillUsedClasses flatCollect: #clientTypes)) asSet;
forEach: [ :aClass |
view nodes
shape: [ :each |
BlElement new
size: 5@5;
background: Color gray ];
with:(aClass methods intersection: interestingMethods).
view layout grid ].
view edges connectFromAll: #clientTypes.
view layout force
strength: 0.15;
charge: -250;
length: 70.
view
The code of this view starts to increase; however we also get a better sense of the way deprecated classes are used. We noticed now that two deprecated classes have many methods called from within the system. However, only few methods call those deprecated methods.
How used are deprecated methods?
In a deprecated class that has many methods, which method should we first look at and try to understand? We could start with those methods that have the most clients; of maybe with those that only have a few? Regardless of our choice we first need to determine the usage of those methods. Since finding this manually can be time consuming we should create an analysis.
For this tutorial, one way in which we can show usage is to change our view to set the size of a method in a deprecated class based on the number of client methods using it. Like this, if a method is large it means that it has many usages in the system.
view := GtMondrian new.
view nodes
shape: [ :each |
BlElement new
padding: (BlInsets all: 2);
constraintsDo: [:c |
c horizontal fitContent.
c vertical fitContent ];
background: ((each isAnnotatedWith: 'Deprecated')
ifTrue: [ Color lightRed ]
ifFalse: [ Color veryLightGray ]) ];
with: (stillUsedClasses,
(stillUsedClasses flatCollect: #clientTypes)) asSet;
forEach: [ :aClass |
view nodes
shape: [ :each |
BlElement new
margin: (BlInsets all: 2);
geometry: BlCircle new;
size: ((each parentType
isAnnotatedWith: 'Deprecated')
ifTrue: [ (each clientMethods
intersection: clientMethods) size + 5 ]
ifFalse: [ 5 ]) asPoint;
background: Color gray ];
with:(aClass methods intersection: interestingMethods).
view layout rectanglePack ].
view edges connectFromAll: #clientTypes.
view layout force
strength: 0.15;
charge: -250;
length: 70.
view
The view above shows us that two a few deprecated method are used more; also these seems to be no deprecated method used much more than the others.
Let’s try a different system
Before we wrap up let’s see how this analysis would look like if we apply it on another system. For this tutorial that system is Apache Lucene Solr. This is a much larger system than ArgoUML having 11k classes and approximately 1M lines of Java code. For the tutorial we use the version corresponding to the commit hash 52f2a77 .
If you are following the tutorial in Glamorous Toolkit you can redo all the previous step using the predefined archive lucene-solr.zip available at https://dl.feenk.com/moose-tutorial/lucene-solr/lucene-solr-52f2a77.zip (the archive is about 230 MB). Next we focus only on a few steps.
Initially after loading the model we can again see details about the system.
We can apply the initial view showing dependencies between deprecated classes. We see a very different situation than in the case of ArgoUML.
Again, to extract value, we need to customize the view to this system. As the name suggests the system is composed out of two subsystems: Lucene and Solr. Let’s add a blue border to classes belonging to Lucene and a green border to classes from Solr.
view := GtMondrian new.
view nodes
shape: [ :each |
| containerName borderColor |
containerName := each container mooseNameWithDots.
borderColor := Color gray.
(containerName beginsWith: 'org.apache.lucene') ifTrue: [
borderColor := Color blue ].
(containerName beginsWith: 'org.apache.solr') ifTrue: [
borderColor := Color green ].
BlElement new
size: 5@5;
geometry: BlCircle new;
background: ((each isAnnotatedWith: 'Deprecated')
ifTrue: [ Color red ]
ifFalse: [ Color gray ]);
border: (BlBorder paint: borderColor) ];
with: (stillUsedClasses,
(stillUsedClasses flatCollect: #clientTypes)) asSet.
view edges connectFromAll: #clientTypes.
view layout force.
view
We see now several clusters of clients using deprecated classes from Lucene. We also see a potentially more problematic deprecated class that has clients from both Lucene and Solr.
Last but not least, let’s apply the view showing the size of methods from deprecated based on how much they are used in the system.
Now we get a more complete picture of how deprecated classes are used in Lucene-Solr. We see that in this case there are several methods that are highly used. Also client classes only have few methods that use deprecated methods.
We can now use this view as a way to guide the way we read code.
Wrap-up
In this tutorial we explored the creation of an analysis for investigating deprecated classes in Java systems. Even for this relatively small task there are still many other ways in which we could adapt our analysis.
For example, what about deprecated methods used from within other deprecated methods? Maybe we do not want to take those into account. Maybe we care more about the size of methods, etc,. Depending on what we want to achieve different kinds of information become important to us. In all these cases we can go and adapt our queries and views.
Understanding deprecated classes is but a simple example. Software development is filled with many other such problems. And each of these is approachable in the same way: start from the question; craft custom analyses that target specific contextual needs; interpret the results; repeat until you are confident about what to do next. And all this, without reading (much) code. This is the essence of moldable development.
Appendix
This this appendix we look as how to manually create a MSE model file for a system. For this we need three things:
- the source code of the system
- all dependencies of the system
- jdt2famix, a tool that takes the source code and dependencies of a system and creates a MSE file
In the concrete case of ArgoUML we should then do:
- Download and unzip the latest release of jdt2famix.
- Create an
ArgoUML-0-34folder. - Download the ArgoUML version 0.34 sources and place them in
ArgoUML-0-34/srcfolder (the name of the src folder can be arbitrary) - Download the ArgoUML version 0.34 libraries and place them in
ArgoUML-0-34/dependenciesfolder (the name of the dependencies folder can be arbitrary) - Go to the
ArgoUML-0-34folder - From the command line execute
<path to jdt2famix>/jdt2famix.sh .(for windows you can usejdt2famix.cmd)
This will result in an ArgoUML-0-34/ArgoUML-0-34.mse file with the serialized model that we can load and explore in Glamorous Toolkit. In case there are classes that cannot be resolved in the system, they are logged in the file ArgoUML-0-34/jdt2famix-import.problems .
We can follow a set of similar steps for Lucene Solr. In that case we can clone the repository and use Maven to get all dependencies. More information about how to do this can be found on the jdt2famix webpage.
