We need safer APIs and safer implementations

6 min readOct 22, 2018

Both the Scala code and Java code shown below will cause your application to fail at Runtime. It would be safer if it just didn’t compile.

Running this code in Scastie, the online Scala REPL will result in a blow up!

Running the Java code above in Repl.it results in

Ideally neither API would expose a method with a signature that allows us to make an illegal request. Where LinkedLists expose a ‘head’ method, it should only ever appear on the non-empty version of that List. And, in the age of Option (Scala) and Optional (Java) there are better ways of handling out-of-bounds Exceptions than blowing up your application at runtime.

Does a new API make sense?

The JDK Collection API is designed primarily (and is really only suitable) for working with mutable collections. The method signature of the mutator objects require that the Collections be mutated in place, preventing mmutable instances returning a new Object reference. This leaves libraries implementing immutable collections with no choice but to not implement these methods.

The Javadoc (above for List::add) further defines the semantics of the method and requires developers who don’t implement adding an element in this method to throw a NoSuchElementException.

There is marginal benefit to creating a new Collections API inside the JDK for safe access (e.g. via a get method that returns an Optional type that is empty for out-of-bounds requests), but that calculus changes significantly for immutable collections.

What could a new JDK API for ImmutableLists look like?

Cyclops X defines a number of Persistent Collection (immutable collections that make use of efficient shared memory) interfaces with a core set of Collection methods. E.g. for Persistent Lists the core methods look something like this

Unlike the methods in JDK collections the return type of any ‘mutator’ methods such as plus / insertAt / removeValue all return another Persistent List. The accessor methods either return an Option (a Cyclops X optional like type with a safe API that also does not need to throw Exceptions) or accept a default value to return in the case of out-of-bounds access.

It would be nice to see a core set of bare-bones Immutable collection interfaces defined in the JDK that could be resused across libraries (I think this would be worth it, despite the likely cost of Optional being the return type, especially if the unsafe Optional::get could also be deprecated).

Seq : a safe immutable LinkedList

Seq in Cyclops X is a persistent Linked List implementation of this interface. It solves the interface challenge seen in the Scala List by making the head only available on the non-empty version of the List (typically called Cons) e.g.

When Java (eventually) adds Pattern Matching via JEP 305, we may be able to leverage it with code like this

By switching on the list, we can access the non-empty type separately from the empty version.

While we wait for Pattern Matching to be added to the JDK, we can take advantage of folds (or Catamorphisms) to do much the same thing today :-

Laziness = performance? Watch your toes when using power tools!

Making strict (non-lazy datatypes) APIs safe is relatively straightforward. Lazy data types may offer big improvements in performance (the fastest code is always the code that is never run), but potentially open up a new range of safety pitfalls. Unexpected results and infinite loops.

What will the Scala code above print out? Stream in Scala is a lazy LinkedList (an actual data structure) something very different from Java’s iterable Stream of values (in j.u.stream.Stream). I think most people might reasonably expect this code to print out (as Stream is lazy and no value is ever accessed)

“!”

But it doesn’t. The value of b ends up being changed to the first value in the Stream.

This behaviour will be fixed when Stream is replaced with LazyList in Scala 2.13

Similarly we might reasonably expect the code below to return true, after all, the Stream should be empty.

In fact this results in an infinite loop. Running this code in Scastie we get a timeout error.

The API isn’t at fault this time, it is not what is preventing the implementation from behaving correctly. Instead it is the implementation itself.

LazySeq : a safe Immutable Lazy List

LazySeq in Cyclops X is a Lazy Immutable LinkedList. The Java code below, does do the right thing in that the variable b is never mutated because LazySeq is truly, fully Lazy.

The output from Java looks like this

Similarly the Java implementation of the Scala code that caused an infinite loop, also runs correctly.

The Java output is like so :-

What’s wrong with Optional?

We are probably stuck with Optional::get at this stage

and we all know what happens next

But by 2020, hopefully with Cyclops’ safe Option (or fingers crossed an upgraded JDK Optional) we are able to write code like this instead

In the meantime in Cyclops X we can write code like this

Safe APIs and Implementations

Reducing runtime and logic errors in our code involves making sure illegal states are unpreventable in our APIs, but also ensuring that our implementations first and foremost do what our users might reasonably expect. In the ideal situation if you can write it, it should run.

It is unfortunate that calling collect a second time on a Stream results in another Runtime Error

An IllegalStateException