Designing a Lightweight Plugin Architecture in Java

Suvodeep Pyne
Jun 17 · 6 min read

Do you want to make your Java app extensible and easy for others to add functionality?

Are you struggling with OSGI and are looking for a simpler solution?

Are you in a classpath hell because your app talks to too many things and everything sits in your classpath?

In this article, I’ll try to offer some solutions to these problems often encountered by folks, as the application grows through its lifecycle.

When do I need a Plugin architecture?

A plugin architecture might make sense for the following set of problems

  • Your app offers a certain set of functionalities and it also interacts with a bunch of other projects
  • Different parts of your app does different things which are unrelated to each other but they interact well defined contracts
  • You need classpath isolation. Your app has conflicting transitive dependencies which is becoming increasingly difficult to manage.
Bad dependency situation

Other Solutions

There are various approaches that are used in practice to deal with such issues.

  • Shading
  • OSGI

Shading alleviates some of the dependency issues by renaming packages. However, it creates multiple versions of the same package and could lead to a debugging nightmare.

OSGI was probably the industry standard in building modular Java apps but for most developers, it is too heavyweight and hard to use.

What people say on Stack Overflow

“OSGi brings more pain than benefit”

“My personal (2 years) experience with OSGI is that the technical bill outweighs functional benefits by orders of magnitude. I have encountered cases were you would have to create/edit 25+ pom files to implement a one liner mock!”

Writing your own infra

There are a few steps in building a simple plugin infrastructure in Java. To keep it simple, I wrote a quickstart on GitHub to help you get started. The project uses maven and Java 8 but the principles can be very easily extended to other build systems/jdk versions.

We’ll be using Java’s ServiceLoader to do all the heavy lifting for us.

Let’s start with what the distribution/tarball.

You can also do this with an uber jar but you’ll need some sort of a structure to arrange your plugins and their binaries.

Personally, I prefer having a distribution instead of cramming everything in a giant jar.

Distribution

A possible way for setting up your distribution could be something like this:

├── bin # contains your launcher
│ └── launcher.sh
├── lib # contains all jars for your main app
│ ├── java-plugin-quickstart-core-1.0-SNAPSHOT.jar
│ └── java-plugin-quickstart-spi-1.0-SNAPSHOT.jar
└── plugins # dir containing your plugins
└── foo # An example plugin dir
└── java-plugin-quickstart-plugin-foo-1.0-SNAPSHOT.jar

This is a pretty standard structure but let's still go through each of them.

  • bin: contains your launchers, your shell scripts, etc
  • lib: contains jars and resources in your app classpath. (Not your plugins)
  • plugins: dir containing all your plugins. each plugin itself is contained in a sub-directory. Here foo is an example plugin

Packaging

Packaging is crucial as it allows you to define a clear contract as to how your app interacts with its plugins.

  • spi : spi stands for Service Provider Interface. (Earlier folks would call it Serial Peripheral Interface. take your pick! ) spi is intended to be extremely lightweight and contain only the essential interfaces of the app. Ideally no implementations, or code should reside here. Dependencies of this package should be at a bare minimum. The goal is that all plugins and core packages will depend on the spi package and it is the anchor point of the application.
  • core: This is intended to be your app which may be loading the plugins using the spi interfaces and would therefore depend on your spi package.
  • plugin-foo: Your plugin implementation which would depend on just your spi package and implement a set of your spi interfaces offering a certain feature/extension. In most cases, you probably shouldn’t be depending on any other core packages of the app.

Interfaces

Having a set of good and clean interfaces is critical in building a modular app. Here, the goal is to be able to load a service class called Foo. Foo is built by a class called FooFactory which is loaded via a Plugin . Let’s walk through each one of them.

We’ll start with the Plugin interface.

public interface Plugin {default List<FooFactory> getFooFactories() {
return Collections.emptyList();
}
default List<BarFactory> getBarFactories() {
return Collections.emptyList();
}
}

This interface must be implemented by every Plugin. The Plugin interface is used by Java’sServiceLoader to load the plugin and make it available to the rest of the app.

Every plugin can choose to offer one of more factories. So, a plugin is not restricted to be of a certain “type”. The factory pattern used here makes it easy for the app to construct service objects on demand. It also gives complete control to the plugin as to how they want to construct the service instance.

A simple Factory class would look something like this. Here the name defines the kind of object created.

public interface FooFactory {  // Expected to be unique
String name();
Foo build(FooContext ctx);
}

Finally, your main service interface would look something like:

public interface Foo {  void doFoo();
}

Writing a Plugin

A plugin is simply a module that implements the Plugin interface and exposes it to Java’s ServiceLoader . The plugin module would typically add spi as its dependency to be able to implement the interfaces.

java.util.ServiceLoader would read the text file resources/META-INF/services/org.spyne.javapluginquickstart.spi.Plugin to locate the reference to the plugin implementation. In this case, the content of the manifest file would look something like

org.spyne.javapluginquickstart.fooplugin.FooPlugin

Here’s a complete example:

Loading a Plugin

This is where it all fits in. The ServiceLoader interfaces uses a URLClassLoader to load the service. The quickstart extends the URLClassLoader to a PluginClassLoader that may be used to limit access to some of the classes in the main application classloader.

So, the ClassLoader boundaries look like this.

Plugin Architecture

Running the entire thing

You can build the project

./mvnw install

This will build the application distribution in the target dir inside the distribution module which contains a launch script.

❯ bin/launcher.sh
Hello World!
Loading plugin: plugins/foo
Installing plugin: org.spyne.javapluginquickstart.fooplugin.FooPlugin
I'm a foo dooer!

Upon running the launcher , you’ll notice that the application is able to load up, iterate over all the directories inside the plugins dir and install them. Post installation, the Foo service is now accessible to the core module at runtime.

Also note that since the System classloader is different from the PluginClassLoader, the app is mostly insulated from the jars inside the plugin. Not just that, every plugin is working in its own classloader giving each plugin its own class space.

Note that you would still need these plugins to be good citizens inside the app since they all are still running in the same JVM.

Conclusions

With a bit of work, you can easily set up your own plugin infra with Java. This is much cleaner architecture that helps maintain modularity and builds a solid skeleton for the app to stand on.

I’d really like to thank the Apache Pinot and Facebook prestodb team for putting out some really amazing code in the open source community. You can go through both of these projects to see how they leverage classloaders and build modularity in the codebase.

Again, the full implementation is available in the repo shared below.

Hope this helps! Feel free to hit me up on twitter or leave a comment below in case of any questions.

Geek Culture

Proud to geek out. Follow to join our 1M monthly readers.