Handling asynchronous errors in Scala at Hootsuite

Brian Pak
Brian Pak
Aug 9, 2018 · 4 min read

Introduction

Every day Hootsuite makes hundreds of thousands of API calls, and processes millions of events that happened in various social networks. Our microservice architecture, and a handful of asynchronous servers with efficient error handling, make this possible. Let’s take a look at how the Scala servers deal with errors.

Different types of error handling in Scala

First, let’s see what kinds of error handling mechanisms exist in Scala

Exceptions

Unlike Java, all exceptions in Scala are unchecked. We need to write a partial function in order to catch one explicitly. It is important to make sure that we are catching what we want to catch. For example, use scala.util.control.NonFatal to catch the normal errors only.

// Example codetry {
dbQuery()
} catch {
case NonFatal(e) => handleErrors(e)
}

Options

Programming in Java often produces abuse of null to represent an absent optional value and it led to many nasty NullPointerExceptions. Scala offers a container type named Option to get rid of the usage of null. An Option[T] instance may or may not contain an instance of T. If an Option[T] object contains a present value of T, then it is a Some[T] instance. If it contains an absent value of T, then it is the None object.

// Example codeval maybeProfileId: Option[String] = request.body.profileId
maybeProfileId match {
case None => MissingArgumentsError(“ProfileId is required”))
case Some(profileId) => updateProfileId(profileId)
}

Try

Unlike Option, Try can be used to handle specific exceptions more explicitly. Try[T] represents a computation that may result in a wrapped value of type T, Success[T] when it’s successful or a wrapped throwable, Failure[T] when it’s unsuccessful. If you know that a computation may result in an error, you can simply use Try[T] as the return type of the function. This allows the clients of the function to explicitly deal with the possibility of an error.

// Example codeTry(portString.toInt) match {
case Success(port) => new ServerAddress(hostName, port)
case Failure(_) => throw new ParseException(portString)
}

Either

Either is the more complicated but better way to handle errors; we can create a custom Algebraic Data Type to structure and maintain the exceptions. Either takes two type parameters; an Either[L, R] instance can contain either an instance of L or an instance of R. The Either type has two sub-types, Left and Right. If an Either [L, R] object contains an instance of L, then it is a Left[L] instance and vice versa. For error handling, Left is used to represent failure and Right is used to represent success by convention. It’s perfect for dealing with expected external failures such as parsing or validation.

// Example codetrait ApiError {
val message: String
}
object ApiError {
case object MissingProfileError extends ApiError {
override val message: String = “Missing profile”
}
}
def getProfileResult(
response: Either[ProfileError, ProfileResponse]
): Result =
response match {
case Right(profileIdResesponse) =>
Ok(Json.toJson(profileIdResponse))
case Left(MissingProfileError) =>
NotFound(ApiError.MissingProfileError)
}

Asynchronous usage

We have looked at various of methods used for handling errors, but how will they be used in multi-threaded environments?

Future with failure

Scala has another container type called Future[T], representing a computation that is supposed to complete and return a value of type T eventually. If the process fails or times out, the Future will contain an error instead.

// Example codeval hasPermission: Future[Boolean] = permission match {
case “canManageGroup” => memberId match {
case Some(memberId) => canManageGroup(memberId)
case _ => Future.failed(BadRequestException(MissingParams))
}
}

Future without failure

If we review the example code above, one improvement we can make is to not raise an exception for a missing argument. To handle the error in a more controlled, self contained way, we can combine the usage of Future and Either.

// Example codeval hasPermission: Future[Either[PermissionError, Boolean]] = 
perm match {
case “canManageGroup” => memberId match {
case Some(memberId) => canManageGroup(memberId).asRight
case _ => BadRequest(MissingParams)).asLeft
}
}

Simplify Future[Either[L, R]] with Cats EitherT

While it is a good practice to handle errors or perform validation asynchronously using Future and Either, adding chains of operations such as (flat)mapping and pattern matching on the containers can require a lot of boilerplate. EitherT can be used to remove the hassle. EitherT[F[_], A, B] is a lightweight wrapper for F[Either[A, B]]. In our case, Future[Either[L, R]] can be transformed into EitherT[Future, L, R] which gets rid of the extra layer between Future and Either.

// Example codedef updateFirstName(name: String): 
Future[Either[DataError, UpdateResult]] = ???
def updateLastName(name: String):
Future[Either[DataError, UpdateResult]] = ???
def updateFirstAndLastName(firstName: String, lastName: String):
Future[Either[DataError, UpdateResult]] =
updateFirstName(firstName) flatMap { firstEither =>
updateLastName(lastName) flatMap { secondEither =>
(firstEither, secondEither) match {
case (Right(res1), Right(res2)) => sumResult(res1, res2)
case (Left(err), _) => Future.successful(Left(err))
case (_, Left(err)) => Future.successful(Left(err))
}
}
}
// Example codedef updateFirstAndLastName(firstName: String, lastName: String):
EitherT[Future, DataError, UpdateResult] =
for {
a <- EitherT(updateFirstName(firstName))
b <- EitherT(updateLastName(lastName))
result <- EitherT(aggregateResult(firstRes, lastRes))
} yield result

Conclusion

Most Scala services at Hootsuite use all of the error handling patterns mentioned above in appropriate situations. Either is widely used to gracefully control business errors, Try filters expected failure more explicitly, and Option is seen in a lot of places where the value can be absent. The combination of Future and Either is definitely the most prominent, but this can make the code quite noisy due to double wrapping of objects. This problem is solved by adopting EitherT, the monad transformer from the Cats library. It allows us to create clean and readable but powerful asynchronous code.

Hootsuite Engineering

Hootsuite's Engineering Blog

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store