Plotly.kt: a dynamic approach to visualization in Kotlin

Kotlin likes plotly

A comfortable data visualization tool is what Kotlin ecosystem desperately lacks for scientific applications. While we wait for charts.kt to be released and become at least partially free, we need something to draw simple charts. There are not a lot of ready flexible libraries for charts in the JVM ecosystem. One possibility is JFreeChart. The library is definitely one of the most powerful libraries for charts, but it is quite old and sometimes hard to get around without previous knowledge. It also is not possible to use it in a multi-platform environment.

Most of the modern charting libraries went to the browser/JS wold. In our laboratory, we mostly use the Plotly library, which provides a broad feature set and very good interactivity. Also, it has interfaces with Python and JavaScript, but, sadly, no Kotlin. Well, it could be remedied. Plotly is basically pure JS library, meaning that in order to draw something in it from JVM backend, one needs to either create a static HTML page and send it to a browser or start its own web-server. We need visualization in Kotlin, yet we delayed this work as far as we can since it seemed rather awkward and time-consuming to do.

Plotly.kt

Well, we’ve done it in the end. Please, welcome the plotly.kt library. It is designed as a multi-platform library, yet for now, it is focused on JVM implementation (client-side Kotlin-JS is coming in the next release, please vote for this issue to make it go faster). Let’s explain, how it works.

First, you create a plot object like this:

The layout should be familiar to plotly users, but just in case:

• trace function create a data trace with given arrays for x and y coordinates.
• layout describes general plot parameters like title and legend and axis.

As soon as Plot is created, one can do different things with it. On JVM (meaning Kotlin-JVM target on a multi-platform project or plain old kotlin("jvm")) it is possible to create a stand-alone HTML page file with the plot and show it in a browser. Like this:

plot.makeFile()

The makeFile has two optional arguments. The first one explicitly defines a file path, to which the plot should be saved and the second one defines if the result should be shown in browser immediately.

A picture from script above

The second thing one could do is to display your plot (or several plots simultaneously) in an embedded ktor server. Like this:

val server = Plotly.serve{
    plot(plot)
}
//We need a way to shut server down
println("Press Enter to close server")
readLine()
server.stop()

Now we only have to follow the instruction in the console and access localhost port via browser to view our page. The served content could be much more complicated than that and include multiple pages with multiple plots each. The example could be found here.

An obvious question is why would we want a server if we already could generate a static page. The answer is rather obvious: we want to change the plot after it had been created. The server is able to update the plot dynamically as soon as the data or any other parameters are updated (for now this option must be turned on by configuration flag):

In this case, we create a trace in advance and remember a link to it, then we publish our plot and start to change data regularly using asynchronous updates with kotlin coroutines. You can try it and see that the plot starts to update data rapidly. The same could be done to any trace or layout parameter.

Currently, the library supports only the basic functionality of Plotly, but do not despair, you can use any feature without waiting for the library update (yet it would be very helpful if you opened an issue for the feature you want to be implemented). For example, you could want to add labels to data points:

It is easy. You can use configure function to add any dynamic property to your objects. Just open Plotly documentation and chose. We plan to add new statically-typed features as soon as they are requested. Custom features also support dynamic update!

Internals

If you ever worked with similar problems, you probably already noted that some things I showed earlier are quite easy and some are much harder to do. For example automatic update for any property (including custom ones), custom properties themselves and dynamic update of HTML page without constant reload. This article is made to explain how it works inside.

The trick where properties update something when changed is usually called reactive properties and has several implementations. For example, famous JavaFX properties which could be bound to other properties and have listeners. In JavaFX, there are special property classes that have all of these complicated binding mechanics inside them. It is quite useful when you are working with the framework already designed to work with it (like JavaFX UI), but quite hard to do from scratch. One needs to create a reactive property for each class field and then duplicate it to allow simple user-friendly access. Like this:

val property = SimpleStringProperty("a")
var string by property // this is tornadoFX access delegate

Also when the property changed you need either to invalidate the whole structure or invent some clever way to pass partial changes.

In the case of Plotly.kt we used a different approach. In fact underneath, the structure we use to describe traces or layouts is dynamic (not unlike JSON tree or map of maps)! We use DataForge-meta metadata processing library core, which gives a convenient tree-like structure called Meta for description. The Meta has a mutable flavor which allows to attach listeners to different elements and could be easily converted to JSON to be transported to Plotly.js client. That is where the custom features came from.

But what about the statically typed objects we use for builders? How we got them? It is time to feel the full power of Kotlin. Let’s see the listing for a Layout builder class:

You can see that the class looks almost like a regular class with modifiable fields. But in fact, it is not. No fields here. All of the properties are in fact delegates that from the user point of view looks like typed properties, but in fact, just change the state of the dynamic Config object passed as parameter ( Config is a mutable Meta the descendant with some additional benefits). The whole class is a specification decorator for an untyped dynamic object. When you finish changing it, you can just strip the specification, transform the inside Meta tree into JSON and send it to a server without creating any data transport object. The same could be done in the opposite direction. You can receive the object, check it and wrap it in the specification without making a copy. In terms of context-oriented paradigm, the specification generates a stateless context, where Meta tree acquires type-safe behavior (this behavior is not available outside of the context). The underlying object could even have additional fields not known to specification, so you can also forget about problems with synchronizing client and server versions when you add additional fields. If one them does not know about the new field, he just ignores it. The dynamic nature also allows adding very simple listening mechanics. You do not have to listen to each property and and custom reaction to it, you can listen to tree change in a universal way.

Of course, nothing in this world is free. The delegates and tree lookup are less performant than the direct access to class members, meaning that you probably should not place specification read or write in a performance-critical part of your code. Still, for configuration and UI updated it is quite acceptable.

Bye-bye reconcile

The algorithm I’ve shown should be rather familiar to JavaScript developers. Specifically to ReactJS users. The idea is the same, you have a dynamic tree, then you change it and pass the modified tree to your UI framework to evaluate. But there is a difference. In React you seldom pass not only changed parts but the whole modified tree and then the internal algorithm called reconcile decides which parts of the UI graph should be changed and which parts are not by comparing the state tree with the incoming change tree. It is usually necessary because the server generates a full UI description from a statically typed object.

Here, the behavior is different. We do not need to pass the whole scene tree to the server. Each change of property generates a meta change event that knows the exact name of the modified property and its new value. So we can send only changed value without the requirement to calculate the difference. If the changes are rapid and one wants to save on packages, it is possible to accumulate changes in a specific local Meta tree and send them all together only after some minimal time (this is actually implemented in plotly.kt server). If changes cancel each other, they won’t be sent at all.

What else?

For now the library supports server and file display on JVM, but in the next release it will be possible to use it directly on Koltin-JS. In future it should be also add a native flavor without a lot of effort.

We also plan to gradually add a support of different dash widgets. Feel free to leave your feature requests and remarks at the issue tracker.