Which Java exceptions are the most frequent?
Samebug has a large collection of crashes from the web: stack traces with a web page, where we found it, attached. We grouped these crashes based on a number of criteria: exception type, software component throwing the exception, main sources where we found them etc. We’ve also identified common error patterns (the part of the stack trace that is the same in several different stack traces) which cause lots of trouble for developers.
We then started to analyse the data to gain some insights into the most frequent Java exceptions. We looked at 3 measurements: the number of collected crashes, the number of web pages where a stack trace related to the exception appeared and the number of error patterns identified by our algorithm. Since at Samebug we are working on creating a platform which helps fix bugs easily, such an analysis was essential. We solve bugs for which it is absolutely necessary to know which ones occur most frequently so we can be effective. Here is what we found out from the analysis:
When looking at the number of crashes we can see that, not surprisingly, the NullPointerException is by far the most common error. It can also be found on the largest number of web pages among the exception types as we can see in diagram 2:
The difference between this and the previous one is that a web page can contain several different stack traces (e.g. multiple stack traces pasted into a single GitHub issue). We can see that there are a few differences in the order of the exception types: e.g. IllegalArgumentException and IllegalStateException have a more notable presence in discussions on the web when compared to their importance measured only in the number of crashes.
In diagram 3 we ranked exception types according to how many error patterns are related to them. It can be seen that some exception types rank lower than in the first graph. This means that less patterns are connected to them but those patterns have more crashes on average.
The most remarkable such type is the ClassNotFoundException. In fact, when we worked on grouping error patterns, we’ve already noticed that the patterns most commonly found in different crashes, in other words which are shared by numerous different developers, are related to ClassNotFoundException. We‘ve written about how to handle ClassNotFoundExceptions before so let’s compare its crash/error pattern distribution to that of the NullPointerException. We found that many ClassNotFoundException patterns have a huge number of crashes in our database. Meanwhile in the case of NullPointerException there are much more error patterns but these collect significantly less crashes on average. We’ve illustrated this in the diagram below (note that the numbers are on log scale).
In the case of NullPointerException there are more than 30,000 patterns each with less than 10 connected crashes and 65 patterns with 100–999 crashes. In the case of ClassNotFoundException there are only 10000 patterns with less than 10 related crashes but there are lot more patterns with more than 100 crashes when comparing to NullPointerExceptions. There are even patterns with 1000+ connected crashes and one pattern has more than 10000 connected crashes.
This result may come as a surprise to many so let’s find out the reason behind this:
Calls in the code which lead to a crash with NullPointerException are quite dispersed because they come from many different libraries and take effect immediately when one makes a mistake, while the ClassNotFoundException is generated by a standard mechanism of the JVM and therefore leads to many stack traces with the same error pattern.
One could think that both NullPointerException and ClassNotFoundException are errors so basic that every developer knows how to solve them easily but their frequent occurrence on forums contradicts this assumption so writing a guide on how to handle them is definitely helpful for a lot of developers. In the case of NullPointerException it is hard to write a few general solutions because the problem is different in each library. In the case of ClassNotFoundException there are some error patterns common to a lot of developers. Of course there are specific solutions for different environments or frameworks you use but the general solution provided is still meaningful.
We performed this analysis so we can choose smartly which patterns to start with. We came to the conclusion that ClassNotFoundException error patterns should come first even if there are more crashes with NullPointerException.
We are currently looking for experts in Neo4J, Java Runtime, MySQL, Google Guava, JUnit, Tomcat, Hibernate, Spring, Android to write short summaries and solutions for error patterns connected to these components. If you are interested, reach out at hello@samebug.io.
