Pointer is a reading club for developers. It’s a window into what other current and future CTOs are reading and thinking about.

Subscribe at www.pointer.io

Building a Startup with JavaScript and Scala

by Alex Poon

Author’s Background: Alex is the COO and Founder of x.ai, an artificial intelligence powered personal assistant. He has over fifteen years of experience in data analytics, software development and product management. Alex was VP of Engineering at Outbrain after it acquired the company he co-founded, Visual Revenue. Earlier in his career, as an engineer at Lockheed Martin, he built software for unmanned aerial vehicle and radio/wireless signal processing. Alex received his MBA from Columbia Business School. He also holds a MS and BS degree in Electrical Engineering. Zerg is his preferred race at Starcraft II. You can follow him on Twitter @alexpoon06

One key strategy to maximize the chance of success when building a startup is to minimize the amount of time re-inventing the wheel. It is about picking the right tool for the job and innovate only in places that add lots of value. To build an AI-powered personal assistant at x.ai, we decided that both Scala and JavaScript are the right languages for us. Having two languages in production certainly adds complexity. IDE, code deployment, libraries, tools, etc are drastically different between languages. Not to mention our engineers’ skills and knowledge in both. Despite all these disadvantages, we embarked on our journey with them and haven’t looked back.

Machine Learning with Scala

When we started x.ai, Python and Scala both have particularly good support for and strong community in machine learning and NLP. Having built our last company’s backend in Python and myself being a Python guy, Python was a strong contender. In terms of libraries, there’s a great list including NLTK, Scikit-Learn, Panda, and Scipy. It is also more likely that a data scientist coming directly from school would have experience with it. In the end, we picked Scala for its strong type system, suitability for functional programming, and the expressiveness of the language. I recall some of the nastiest bugs that hurt us most when using Python were rooted from mutable objects being changed in unexpected places or some downstream systems disagreeing on Types. I am so excited to be working with a language that is designed to make it easy to deal with these issues. It is designed with side effects in mind and provides the functional tools to handle them. Having Scala power our backend is the appropriate choice for us to implement the complex business logics and artificial intelligence algos in production.

Internal Apps and Third Party API’s with Node.js

We are not building an AI in a black box though. Amy (the name of our AI) “talks” to the world in various ways. She needs to access people’s calendar, receive emails sent to her, etc. Many APIs simply have better support and binding for JavaScript than Scala. It doesn’t hurt that our CTO Matt Casey (https://github.com/mattcasey) knows JavaScript like Snoop Dogg knows weed. We particularly enjoy the Node.js ecosystem with tools like npm package manager to help us coordinate our various dependencies/libraries. Angular.js, the javascript framework, makes our lives much easier when building our web apps. We like the abstraction it provides over HTML, CSS and JavaScript with directives and the modularity that is possible.

Dividing up the jobs

We divide our system across Scala and JavaScript according to a few high level principles. JavaScript is used when we are interfacing with external systems. It is also used for building internal web apps and web servers. We take advantage of tools like Node.js and Angular.js to bring up these services very quickly. Don’t get me wrong, there are plenty of Scala friendly web frameworks to choose from including Lift, Play, Scalatra, etc. We just think JavaScript shines in these areas hands down. Any complex business logics or machine learning/NLP applications needing lots of processing are built in Scala. We have separate JavaScript and Scala teams although some of us are proficient in both languages.

Talking to Mongo…

Database access is specially challenging to deal with having two languages. MongoDB in many ways has been great for us as our primary database (topic of a separate post http://www.slideshare.net/mongodb/building-an-an-ai-startup-with-mongodb-at-xai). Because it is schemaless, we do need to take care to maintain schema across languages at the application level. In addition, we wanted to centralize access to a limited set of endpoints. To accomplish these, we added an internal API layer for production data access. The API Layer is implemented with Mongoose, a very nice object modeling and query library for MongoDB built with Node.js. All key schemas of x.ai are explicitly described in Mongoose. Allowable values for each field and our indexing strategy are described in these files when appropriate. We also push many of the application level joins to this API layer. For documents that have references to others, we implemented “fill” functions that allow us to easily populate the full documents when needed.

It hasn’t been all smooth sailing since we started x.ai with JavaScript and Scala, but we are quite pleased with our decision so far. Having Node.js and Mongoose enabled us to build web apps quickly and create small servers to talk to other APIs. Scala enabled us to take advantage of the vast selection of machine learning libraries, optimize for functional programming and leverage a strong type system. We are able to take advantage of the strengths of both languages, allowing our engineers to spend more time on the fun and challenging stuff — building an AI that the world has never seen before. If this challenge is exciting to you, find out how you can be part of it. https://x.ai/jobs/