Using Scala to Build an AI-Powered Personal Assistant: x.ai Engineers Tell All

x.ai makes an AI-powered personal assistant that schedules meetings for you. Amy Ingram (and her brother Andrew) have processed millions of scheduling related emails so far. The team building Amy is 60 people strong. Among them, 25 data scientists and software engineers created the Scala-based technology that allows Amy to understand meeting scheduling emails and automatically take an appropriate action.

Scala is not new but it’s starting to become more widespread at both startups and larger corporations. We’ve arrived at something of a tipping point: Many more companies are realizing that Scala offers a historically unparalleled opportunity to build software that scales well and behaves correctly. (And not making mistakes is crucial for AI.)

x.ai’s engineering team is excited about Scala and typed functional programming and hopes to advance the tool set. Here’s what x.ai engineers Chris Vogt (@cvogt) and Miguel Iglesias (@caente) have to say about it:

Q: What is Scala (for the uninitiated)?

Miguel: Scala is one of the best compromises out there between industry’s needs/capabilities and academic research (regarding functional programming and type theory). Scala encompasses a lot of the advances from the last 40 years, along with the “hacks” that make commercial use possible. I see it as a very important bridge. But, instead of taking you to the other side, Scala “becomes” the other side, since Scala is evolving as well. The community is getting very good at keeping both concerns — industrial productivity and scientific research — in balance. Of course there are other functional programming languages gaining adoption, but I still think Scala has gotten closer to the “sweet spot,” if there even is one.

Chris: Scala provides developers with powerful tools, strong safety nets, and a lot of convenience. As a result, they feel more in control, are more productive, and are ultimately happier than when using many other languages. Scala’s support for object-oriented programming, a widespread technique, allows for immediate productivity. By providing access to functional programming, which is growing in popularity, Scala lets developers solve many problems, such as scaling, in a much more sustainable way. Scala unifies the two techniques into a largely seamless experience. On top of this, Scala provides a powerful type system and type-inference, which in combination enable developers to easily verify many aspects of correct behavior and just as easily change it. This means more time spent on the product and less on QA.

Typed programming is currently the most established way to analyze what programs do before they actually run. Scala pushes the boundaries on this front in industry.

Functional programming is a technique known to academia since the 1940s but it has never been successful in the industry at scale, until recently. The distributed nature of things today has made the established techniques (in particular, object-oriented programming) almost impossibly hard for some use cases.

Q: How were you introduced to Scala?

Chris: A grad student who had studied with my professor joined the Scala development team. I looked it up, saw that it met needs I had for a long time, and then joined myself.

Miguel: I had wanted to use functional programming for a long time, but was too busy making a living in Java. Working on startups, I had no time for side projects. Then Scala came along, which let me code in functional programming alongside Java files.

Q: What did you find most appealing about Scala? What problems did it solve better than other programming languages?

Chris: Scala enables statically typed functional programming without forcing you into it. So you are always productive, but have the means to write very robust, maintainable code. Good support for immutable data structures is also a game changer. Python and Ruby aren’t statically typed. Go’s and Java’s static typing is quite limited. Java’s syntax is very verbose. Go is weak on re-usable code. Haskell is statically typed and functional, but forces you into it and still needs to prove itself as an industry-ready language for a larger audience.

Miguel: Its depth. Scala seems to have no end. I think you can be very productive in any other programming language, some more so than Scala, some less so. But Scala is very “alive.” There is a lot of stuff going on, exciting stuff, like shapeless, Cats, etc. Its users tend to be very critical of the language and the community itself, which makes it harder to find the kind irrational zealots you’d find in other languages.

Another great thing about Scala is that it brings the benefits of functional programming to the Java Virtual Machine (JVM), which is already an ubiquitous “interpreter.”

Q: Why did x.ai decide to use Scala? What are its advantages in this particular case?

Chris: Some early employees and advisors had experience with Scala and recommended it. You can use the Stanford NLP stack with it, which we used early on. So the decision was made. The advantages are not specific to x.ai, but rather in the general nature of the language.

Miguel: x.ai started using Javascript for its data annotation tasks, but Javascript obviously wasn’t appropriate for large scale data sciences purposes. Python was an obvious choice, since so much data science related work has been done in Python. But all the founders came from a Python shop, where maintaining a large codebase had been very painful. They looked around for statically typed languages, and Scala was an easy choice. You have the good stuff from Python, and then some, plus a good type system. I was very lucky to be the first Scala engineer.

Q: What are Scala’s limitations?

Chris: Scala’s origins in academia allowed it to ship with many outstandingly awesome features, but it also had some rough edges. There are syntax quirks and parts of the standard library that are barely maintained. Also the tooling for Scala is not as polished as it is for many other languages, despite having come a long way already. A bigger problem with Scala at x.ai is actually its much less developed ecosystem of machine learning libraries compared to Python, which is used by many reasearchers.

Miguel: Scala compilation times are horrendous. Scala is trying to integrate features that can be found in languages like Haskell or Ocaml and using the JVM to do it, but JVM wasn’t built with that in mind. That means tools like Integrated Development Environments (IDEs) are impossibly slow, once you start to use advanced features of the language, like type classes and type level programming.

Q: What tools would you like to improve or rebuild altogether to make Scala more usable?

Chris: A better build system (see my NEScala 2016 talk). Less verbose syntax (even though it is already a dream compared to languages like Java).

Miguel: Better compilation times, apparently this will be addressed in Scala 2.12. I’m also very excited about the Dotty project, and the techniques that it could help develop.

Q: How do you see Scala evolving in the next 5 years?

Chris: It seems like adoption in industry is growing rapidly with occasional frustration about the academically-rooted rough edges. I expect the ecosystem of libraries to grow, improve, and mature. The language itself will probably not change very much, except for polishing what’s already there. What Scala probably needs is to back up it’s outstanding features with more polished foundations.

Miguel: I have high hopes in the work happening in shapeless, Cats, and Dotty. The language itself is likely to preserve its syntax, but its theoretical foundations seem to be getting stronger. It’s true that Scala has an academic origin, but it was industry oriented since day one, with all the necessary compromises required in the name of practicality. This means we now have the “foot in the door” of industry, to move the software development into a more “correctness driven” phase.

Q: What new problems might Scala take on that it hasn’t yet?

Chris: Scala is relatively new, but recently received an incredibly good JavaScript backend, which will hopefully conquer more market share, just like Scala on the JVM is already doing.

Miguel: The biggest issue with Scala is it’s biggest asset — it depends on Garbage Collection(GC), i.e. the programmer can ignore memory allocations. The GC takes care of unused objects and frees the memory. This makes Scala unsuited for real time systems. I would love to see Scala, or a descendant of Scala, that’s capable of compiling native code, while staying safe and easy to work with. We know it’s possible. Haskell does it. OCaml does it. I wouldn’t want Scala to move away from the JVM. I’d just love to see it run independent of a third party platform.