This feature comes up when talking about upgrading Scala versions. In this post, I will try to explain what it means and why should we use it.
The goal of this post is to help you update your application to the latest scala version (2.12, 2.13 and counting), And by the end of this post you will know how to answer the following questions :
- Updating scala version — What’s the catch?
- How to cross-compile my libraries?
- How to update the Scala version in my application?
Updating Scala version — What’s the catch?
My first task in BigPanda was to upgrade our Scala stack from 2.11 to the (already quite old now) 2.12 version.
As I learned in this task, Scala is a version sensitive language, which means that code breaks between versions and one cannot use a 2.11 library in a 2.12 application.
Even more, I found that most of the artifacts we define in our
build.sbtare defined using the
%% syntax like this
groupID %% artifactID % revision
which translates to :
groupID % artifactID_scalaVersion % revision
This means, that our application requires that all it’s dependencies to be published in the same Scala version that it’s trying to use.
This process of updating to a new Scala version takes some time for libraries(see the famous spark 2.12 support here) but the real problem lies in the in house dependencies which we have to update first.
How to cross-compile my libraries?
My initial and naive approach was: let’s upgrade our libraries to 2.12 and update all applications to 2.12 also, but it did not work.
The reason for the failure was in the numbers: in BigPanda, we’ve got about 10 scala libraries and 10 complex applications that consume them. Migrating all of them at once is an insanely complex and risky operation that requires complex roll-back strategies and a large amount of time which we did not have.
The more practical way of doing this is by compiling our libraries to both 2.11, 2.12 using a feature called cross-compilation available in SBT.
It’s as simple as adding this line in your build.sbt
crossScalaVersions := Seq(“2.11.11”, “2.12.7”)
And changing your build\publish command from
sbt ++2.11.11 publish
This tells SBT, to publish it twice using the versions in
crossScalaVersions . More info can be found here.
How to update the Scala version in my application?
Now, this is what we really wanted to do. We choose the least critical service in our pipeline and decided to start there and iterate.
The process is as simple as writing this in your
scalaVersion := "2.12.7"
And then making sure all regression tests pass (which is often not as simple).
After we deployed that service into production without any glitches, we felt much more confident about updating the rest of our services.
The happy ending
To be honest, it has been almost a year since we have done this task. we are currently at 9\10 applications migrated to 2.12 and we did not yet reach the part where we delete the cross-compilation into 2.11. It’s not an optimal ending, but a happy one in my perspective.
- Don’t be afraid to update your Scala version!
- Cross-compiling in SBT is a super cool feature that works out of the box and provides the ability to update your libraries to a new version while maintaining backward compatibility.
- Hope for the best, plan for the worst. Migrating all your services and libraries at once is a complex job with a lot of uncertainties.
- I recommend cross-compiling your libraries first to the required version and then migrating the applications one at a time in an iterative manner.