Could machine learning get its own new programming language?

Stefan Karpinski, a research engineer for the Moore Sloan Data Science Environment at CDS, explores the prospect of a new programming language for machine learning

Machine learning research requires a high degree of computational capability and domain specific features from programming languages. Traditional languages like Python scale poorly to these high computational demands, and Python’s semantics complicate some machine learning objectives.

TensorFlow, Google Brain’s open source software library for numerical computation using data flow graphs, is a relatively new programming language that can satisfy domain specific machine learning requirements. But TensorFlow itself involves a type of meta-programming that relies on Python — which means that the code now has two execution times. (And, debugging becomes much more challenging due to differences in language semantics.)

So what programming language can best support machine learning research? Stefan Karpinski, along with a team of researchers, proposes developing an entirely new programming language for machine learning and argues that machine learning is a numerical programming languages problem.

While Karpinski and his collaborators acknowledge that popular numerical languages like TensorFlow can be useful for machine learning, they suggest that a new tailored language would better enable recursion, maintenance, production system integration, abstraction, and program manipulation. The team anticipate that a new language for machine learning would draw inspiration from numerical computing so a user could add data representations in high-level code. They also point out that domain specific language development is not unprecedented — new languages have proven effective for formal reasoning and verification and cluster computing.

But there are significant challenges ahead, such as achieving generality alongside performance, and the requirement of a new library ecosystem.

To meet these challenges, the researchers encourage the machine learning community to draw from the programming language and automatic differentiation communities. They also suggest that the Julia language, a project to which Karpinski contributes, will prove a useful launching pad for future machine learning language research.

By Paul Oliver