Must(read) on Optional(type)

This article is long time coming. It culmination my 4 year struggle to understand proper usage of the optional type.

piotr szybicki
Apr 4 · 9 min read

The optional type that I’m going to talk about was introduce in Java 8 to solve a very specific problem. Some methods, mostly around streams API, had a uncertain result ( I’m talking about findFirst() and findAny() ) that uncertainty if left unchecked would brought a ghost of Tony Hoare us (Null References: The Billion Dollar Mistake). I image with time we could get use to syntax like this:

List<String> numbers= ImmutableList.of("ONE", "TWO", "THREE");
String numberThatImLookingFour =
numbers.stream()
.filter(number -> "FOUR".equals(number))
.findAny();
if(numberThatImLookingFour != null){
return numberThatImLookingFour.toLowerCase();
}else{
return "not found";
}

But the above code is full of cutter and more than a few people would write that code like this:

List<String> numbers= ImmutableList.of("ONE", "TWO", "THREE");

return numbers.stream()
.filter(number -> "FOUR".equals(number))
.findAny()
.toLoweCase();

Resulting in too many NullPointerExceptions. We needed something better. So Brian Goetz and Steward Marks (Java Language Architects) got together and wrote the following paragraph:

Our intention was to provide a limited mechanism for library method return types where there needed to be a clear way to represent “no result”, and using null for such was overwhelmingly likely to cause errors.

I will come back to this quote later. And so the Optional<T> was added. It wasn’t really a new concept (Guava had optional) I think the history goes back to a Maybe monad from Haskell. And suddenly we got JDK approved way of representing value that might or might not be there. And as usual, developers went nuts, using the new capability everywhere. To a point that is a good thing as it allows is to figure out the boundaries of an idea. And I really was pushing it. The Null reference, for me is the source of uncounted bugs, and generally not really a good representation of non existence. The big part on how we should deal with the problem comes from DDD. Where the at the entry to the bounded context we say: ‘You shall not pass

Meaning at the entry to the domain all the variables that are needed to execute business logic are present. But that help us only at reducing the amount of if(obj==null) that we have to write. We still have to interact with the outside world (database queries, rest endpoints etc..) and execute logic based on the output of thous interactions. There, if used properly Optional can help.

What‘s’Optional?

  • It is box type, holds reference to another object.
  • Is Immutable,and non serializable
  • Has no public constructor
  • Can Only be present or absent
  • Created via: of(), ofNullable(), empty() static methods.

How can we retrieve the value from the box?

  • get()
  • orElse()
  • orElseGet()
  • orElseThrow()

There is a catch as temptation to call get() to get the value. We all know getters/setters :). And there is a expectation that if we call get…() we get something. You never expect any exception to be thrown when calling getter. However if you call get on optional and that optional is empty, exception NoSuchElementException will be thrown. Brian Goetz went on a rant during his interview https://www.youtube.com/watch?v=Su1ijHLvuNE how this was a big mistake :). And the methods should be called getOrThorwSomeHorribleError(). Out of that comes the first and second rule:

#1 Don’t assign null to Optional.

I’m not even going to comment on that. Just remember every time you do this little bunny dies. And it’s probably this one:

#2 Avoid use Optional.get(). And never, ever call get() if you can’t prove that optional is present.

Usually we can achieve what you want with orElse(), orElseGet(), orElseThrow(). So we can refactor pattern like that:

String variable = fetchSomeVaraible();
if(variable == null){
1. throw new IllegalStateException("No such variable");
2. return createVariable();
3. return "new variable";
} else {
...
100 lines of code
...
}

into:

1. 
Optional<String> variableOpt = fetchOptionalSomeVaraible();
String variable = variableOpt.orElseThrow(() -> new Exeption(""))
... 100 lines of code ...
2.
Optional<String> variableOpt = fetchOptionalSomeVaraible();
String variable = variableOpt.orElseGet(() -> createVariable())
... 100 lines of code ...
3.
Optional<String> variableOpt = fetchOptionalSomeVaraible();
String variable = variableOpt.orElse("new variable")
... 100 lines of code ...

There is a little caveat that I should mention here. The orElse(..) is evaluated eagerly. Meaning the following code

Optional<Dog> optionalDog = fetchOptionalDog();
optionalDog
.map(this::printUserAndReturnUser)
.orElse(this::printVoidAndReturnUser)

Will execute both methods if the value is present and only the last one if the value is absent. To handle those cases we have method orElseGet() that takes a supplier as parameter. And is evaluated lazily.

#3 Don’t use optional in fields, method parameters, collections.

As part of my own craze for optional I use them in all, above mentioned, cases. This is a bad idea for several reasons.

One it make the code more messy. For example I had to wite one of two setters with either logic in them (code smell, it breaks the contract of the setter, which states that what I set is what I get)

public void setThatField(ThatFieldType thatField){
this.thatField = Optional.ofNullable(thatField);
}

or I had to force all my callers to pack the object in to optional (Maybe less smelly, but also not nice)

public void setThatField(Optional<ThatFieldType> thatField){
this.thatField = thatField;
}
...
setThatField(Optional.ofNullable(thatField));

It led to many problems, many misunderstanding and many bugs.

Two it is also a performance problem. You create extra object. So there is increased GC pressure. And you create extra level of indirection.

Disclaimer: My description is massive simplification. I’m thinking about writing how java is executed on modern hardware but it will take me some time.

So what are those level of indirection. In the end if we want to operate on a value, that value has to find itself in the CPU registers but the registers can only talk to the L1 Cache. If the value is not found in the L1 Cache (here skipping some steps) the load instruction is issued into main memory. We call it a cache miss. In modern hardware that instruction takes around 300 clock cycles. So the following code will take around 900 clk.

object.getListOfObjects().get(0);

We have three references that we have to fetch from main memory. First to get object then to get the list and then to get first object in that list. And the bad part: there is not way to cheat here CPU can’t guess the address and you can’t make it run in parallel. With performance improvements to modern CPU like speculative branch prediction, out of order executions etc. CPU runs from one cache miss to the other, and gets bored in between. Note if you add optional to the mix that means another level of indirection and another cache miss. Let’s say 4 in total. JVM especially JIT can do stuff here to optimize things (escape analysis) but it’s limited. We are waiting for project Valhalla but who knows when that comes around.

Going back, so you can imagine if you have a collection of even modest size, of optionals containing objects that have an optional field you might see a performance degradation pretty quickly.

Also don’t pass optional as argument. It just gives you nothing. First of all you can’t be sure that your caller did not violate the rule number 1. But also when you pass some object to the method presumably you want to do something with that object so you will most certainly end up with this code optional.isPresent(){optional.get()}.If you want to execute some logic based on the presence or absence of that parameter move that burden to the caller. And make him/here choose a method to either create or update some resource. Your expectation (assuming you are inside your bounded context) is that in order to invoke the method, you have all you need..

An exception to that rule could be private methods. Those are usually created to group some functionality together, not part of API surface area and in that case do whatever will result in less code.

#4 Use Optional only as return type. But every time when the result is uncertain.

To be honest this is the only good place to use optional. I will copy paste a quote from the creator of the class himself:

Our intention was to provide a limited mechanism for library method return types where there needed to be a clear way to represent “no result”, and using null for such was overwhelmingly likely to cause errors.

This time I bolded some stuff. This sums up perfectly why to use upotional. Notice that I said don't use optional on fields I said nothing about return from getter :). It is a sacrilege for some people to modify IDE generated getter but it is a way to go.

Here is the great presentation Victor Rentea(Lead IBM Architect for Java)https://www.youtube.com/watch?v=YnzisJh-ZNI&feature=youtu.be&t=847 Where he proposes the same thing. I recommend watching the whole thing, just slow it down to 75% speed, guy has serious ADHD.

#5 Don’t be afraid to use map and filter.

There is more general development practice that is worth to follow called SLA-p

It means Single Layer of Abstraction. I don’t want to get too much off track here but I settle the discussion of weather method is tool long or not (5 line vs 10 lines vs 20 lines) by seeing how many times its body is indented.One indentation is one level of abstraction. Two is two and it’s too much. As our brain is running out of cache lines (we have only 4–7 of those). So the following code:

Dog dog = fetchSomeVaraible();
String dogString = dogToString(dog);
public String dogToString(Dog dog){
if(dog == null){
return "DOG'd name is : " + dog.getName();
} else {
return "CAT";
}
}

can become:

Optional<Dog> dog = fetchDogIfExists();
String dogsName = dog
.map(this::
convertToDog)
.orElseGet(this::convertToCat)
public void convertToDog(Dog dog){
return "DOG'd name is : " + dog.getName();
}
public void convertToCat(){
return "CAT";
}

This is the very simple case but you can imagine how much clearer the syntax become if you can phrase it in this more declarative way.

Filter can be useful fold syntax like this:

Dog dog = fetchDog();
if(optionalDog != null && optionalDog.isBigDog()){
doBlaBlaBla(optionalDog);
}

into this:

Optional<Dog> optionalDog = fetchOptionalDog();
optionalDog
.filter(Dog::isBigDog)
.ifPresent(this::doBlaBlaBla)

And that brings me to the rule number 4:

#6 Don’t create optional just to chain methods on it.

One of the thing to watch out for when using optional is a temptation to chain methods. Things might look so pretty when we chain methods just like in builder pattern :). But pretty not always equal to more readable. So don’t do that:

Optional
.ofNullable(someVariable)
.ifPresent(this::blablabla)

It’s not good for performance, and it’s not good for readability. We should avoid the null references when possible. But when they occur (for example you call 3rd party library) deal with them in familiar way.

#7 Make all expression a single line lambdas.

This is more general rule that I think should be applied to streams as well.But this article is about Optional. In this more declarative style offered us by methods on Optional it is important to remember that the left side of the equation is as much as important to us as the right. But the code will be so hard to follow if we do this

Optional
.ofNullable(someVariable)
.map(variable -> {
try{
return someREpozitory.findById(variable.getIdOfOtherObject());
} catch (IOException e){
LOGGER.error(e);
throw new RuntimeException(e);
}})
.filter(variable -> {
if(variable.getSomeField1() != null){
return true;
} else if(variable.getSomeField2() != null){
return false;
} else {
return true;
}
})
.map((variable -> {
try{
return jsonMapper.toJson(variable);
} catch (IOException e){
LOGGER.error(e);
throw new RuntimeException(e);
}}))
.map(String::trim)
.orElseThrow(() -> new RuntimeException("something went horribly wrong."))

So instead extract all those block lambda to separate methods:

Optional
.ofNullable(someVariable)
.map(this::findOtherObject)
.filter(this::isThisOtherObjectStale)
.map(this::convertToJson)
.map(String::trim)
.orElseThrow(() -> new RuntimeException("something went horribly wrong."));

Summary

That would be it. Optional is not here to replace a null reference it’s here to help us deal with the uncertainty of the result. And thus making the code more readable. And to be honest that is the point when writing code. With a proper usage we won’t pollute the code with a bunch of defensive coding. If you have to fail do it gracefully and don’t try to handle all the transitions of the gigantic state machine that is you application. Remember in the end complexity is our biggest enemy.

12 developer labors

Simplicity is prerequisite for reliability — Edsger W. Dijkstra

piotr szybicki

Written by

Piotr Szybicki’s, Programmer, Java Developer, ML Entusiast

12 developer labors

Simplicity is prerequisite for reliability — Edsger W. Dijkstra