Microsoft open sources its in-memory GraphEngine system
Microsoft on Thursday open sourced a new data-processing system called GraphEngine, an aptly named technology designed for in-memory graph processing. Here is Microsoft’s description of the GraphEngine from the project website:
“Graph Engine (GE) is a distributed in-memory data processing engine, underpinned by a strongly-typed RAM store and a general distributed computation engine.
“The distributed RAM store provides a globally addressable high-performance key-value store over a cluster of machines. Through the RAM store, GE enables the fast random data access power over a large distributed data set.
“The capability of fast data exploration and distributed parallel computing makes GE a natural large graph processing platform. GE supports both low-latency online query processing and high-throughput offline analytics on billion-node large graphs.”
GraphEngine’s GitHub repository offers up a little more information:
“This repository contains the source code of Graph Engine and its graph query language — Language Integrated Knowledge Query (LIKQ). LIKQ is a versatile graph query language on top of Graph Engine. It combines the capability of fast graph exploration and the flexibility of lambda expression: server-side computations can be expressed in lambda expressions, embedded in LIKQ, and executed on the server side during graph traversal. LIKQ is powering Academic Graph Search API, which is part of Microsoft Cognitive Services.”
(Speaking of GitHub, check out my latest podcast interview with Sam Lambert, its senior director of infrastructure engineering.)
We might not expect GraphEngine to take the world by storm, given the relatively limited use case for graph processing and the existence of other open source projects with larger communities, but do expect to Microsoft keep open sourcing more of this type of technology. In its just-getting-started-competition for employees and reputation against competitors such as Amazon, Google and even Facebook, it’s important for Microsoft to show not just that it’s down with open source, but also that it has chops in the world of AI and general big data processing.
We’ve seen Google running this playbook lately with multiple projects, Facebook running it with Beringei and AWS running it with MXNet. Today was Microsoft’s turn.
