The future of in-memory computing

Dr. GP Pulipaka
5 min readAug 12, 2016

In today’s world, the cost of managing the data is high, and the rewards are minimum, as traditional RDMBS tools cannot provide the real-time data customer wants in the now moment. The world is experiencing a faster-emerging big data explosion. Corporations are unable to tap into the right opportunities to deal with the complex conundrums for informed decision-making. Current memory solutions are scalable, however, solutions involve massive investment in the infrastructure. The world needs alternative memory solutions. The main features of future memory solutions consist of power consumption, cost, time to market, density, scaling, and performance. NAND (Not AND), 3D NAND (Three-dimensional Not and), PCRAM (Phase-change memory), STT-RAM (Spin-transfer torque random access memory), and ReRAM (Resistive random-access memory) are the future memory alternatives to DRAM (Dynamic random access memory). The alternatives of using DDP (Dual die package) and TSV (Through-silicon via) in DRAM have cost ramifications. TSV has a projected cost value of 22% higher than DDP.

The complexities for in-memory computing arise from the economies of the flash. DRAM memory used by SAP HANA is an upscale alternative, as opposed to classical RDBMS disk-based systems. Several alternative emerging growth memory solutions explored as part of future memory technologies can replace DRAM.

In 2010, Tokyo Institute of Technology has invented data storage by encoding the data with laser throughout the liquid crystals for permanent storage. CDs (compact disc) and DVDs (digital video disc) can store the data on the surface; however, these only last up to few decades. In 2012, Hitachi heralded that they’ve stored the data permanently on quartz glass plate that can withstand a temperature of 1,832 F. It is also waterproof. Andra, a French nuclear waste management agency is in the process of creating data on sapphire, and platinum discs that can last up to 10 million years.

In 2013, researchers at the University of Southampton in the UK have begun storing the data memory on five-dimensional silica glass discs with femtosecond laser. They’ve demonstrated that, each disc can hold 360 TB of memory. This information can stay forever and does not get destroyed, unlike CDs and DVDs. The researchers at the University of Southampton call this as data that survives the human race estimated to last up to 10 billion years. The transactions in business occur metaphorically at the speed of light, and corporations require an agile system that can process these transactions to stay competitive in the industry. Though, the invention of permanent storage can resolve a great deal of conundrums in the industry to store the data, it does not address the problems associated with speed of the retrieval of the data.

In today’s world, organizations are unable to mine the data to perform real-time analytics, process the information, and harness the power of hardware with economies of scale. Aerospike jumped into the fray of in-memory computing with hybrid memory (DRAM and flash) providing depth of data analytics to spot trends in the business. For a corporation, Aerospike reduced the footprint of servers from 184 to 10. The cost reduced from $2.5 million to $236,000. In October 2013, Aerospike was the only visionary in Gartner Magic Quadrant. Over an extended arc of time business needs a nimble database with ACID properties. Aerospike, the world’s fastest database, recognized once again as one of the visionaries by Gartner in October, 2014. DataStax, the company that delivers Apache Cassandra (a default choice for big-data driven organizations such as Barracuda Networks, Comcast, Constant Contact, eBay, and Netflix) to enterprises recognized as the top visionary by Gartner for 2014.

In the Gartner Magic Quadrant for October 2014, SAP HANA is one of the leaders for ODBMS (Operational database management systems). However, SAP HANA did not secure a position in the visionaries. This provides insights to SAP HANA to revolutionize the infrastructure model to run the database with alternative in-memory options. Gartner hype cycle for emerging technologies 2014 has shown that big data, in-memory databases such as SAP HANA are two to five years from plateauing. SAP HANA was built from the ground up to run entirely on DRAM. The cost of DRAM chips is reducing every year, however, flash memory is highly cost-effective as opposed to DRAM. The future outlook of in-memory databases appears to tilt towards flash memory. In a recent research conducted, results had shown that the cost of data in flash for 2 TB of database is $73, as opposed to data in memory costs $1209.

The cost-benefit ratio is 1:17. Aerospike is already running data in flash successfully with high performance. DRAM in-memory database comes with limitations of scalability, unlike in-memory data grids that can perform massively, parallel processing on extensive, and large sets of clusters of commodity servers. In the same example of the research, in-memory database (DRAM) required 50 nodes 2 TB database requirements. However, data in flash only required four nodes. Currently, many customers globally are performing abstruse assessments to embrace SAP HANA into their organization. The cost is one of the primary concerns. SAP HANA is now slightly above IBM DB2 as shown in Gartner Magic Quadrant for October 2014. However, it could not secure a position in visionaries. SAP should look at the options of economies of flash, and create a hybrid memory with the combination of flash, and DRAM to capture a vast market segment.

SAP HANA for small companies approximately can cost around and under $300,000 with software and hardware. Aerospike offers the database at 1 TB and more, at $7,500 and $75,000 per year. Aerospike recently hit the benchmark of one million writes per second with only 50 Aerospike nodes on Google Compute Engine. This resulted in cost savings by six times. Considering that the cost is a huge factor, SAP HANA requires a forklift upgrade similar to the database Aerospike to run with hybrid memory to provide economies of flash to number of domestic, and global customers.

References

Pulipaka, G. (2015). Big Data Appliances for In-Memory Computing: A Real-World Research Guide for Corporations to Tame and Wrangle Their Data (2 ed.). Los Angeles, CA: High Performance Computing Institute of Technology.

--

--

Dr. GP Pulipaka

Ganapathi Pulipaka | Founder and CEO @deepsingularity | Bestselling Author | Big data | IoT | Startups | SAP | MachineLearning | DeepLearning | DataScience