How HTAP Can Tame the Internet of Things Data Deluge

That’s the total amount of data Cisco forecasts will be pumped out of IoT devices by 2020, up from a whopping 145 ZB measured by the company in 2015 and reported in its Cisco Global Cloud Index: Forecast and Methodology, 2015–2020.

That gargantuan level of Internet of Things data will be thrust into existence by the approximately 20 billion connected devicesGartner sayswe can expect to have ticking away by the end of 2020.

What do we dowith all that IoT data? How can we possibly make sense or use of it? The answer to that question lies, not on the surface, but below it, at the level of the data architecture.

The structure of the architecture needs to allow the data room to scale, because, let’s face it, when it comes to data, less isnotmore. The architecture also has to be reliable enough to allow constancy, because data ideally comes in a strong, steady stream. In modern business, the key to success is applying the best algorithm to the most data to deliver real-time insights. This only works, however, when data read/write speeds are so fast, it’s practically instinctual. And that’s a hard problem to solve for when you’re drowning in oceans of data.

Enter the sexy-soundinghybrid transactional and analytical processing(HTAP) with in-memory-computing (IMC), which has analyst firms practically drooling. Gartner, which coined the term, reckons HTAP will have a “transformational impact on digital business,” while Forrester has bluntly declared that “traditional architectures don’t cut it anymore.” Ouch.

So what exactly is HTAP?

An HTAP architecture supports the needs of many newIoT use casesthat require scalability and real-time performance. It enables instant decision making by bringing transactional data and analytics together at the time of the transaction. An HTAP architecture is best enabled by IMC techniques and technologies to provide analytical processing on the same (in-memory) data that is used to perform transaction processing.

By removing the latency associated with moving Internet of Things data from operational databases to data warehouses and data marts for analytical processing, an HTAP architecture enables real-time decision-making and situation awareness on live transaction data as opposed to after-the-fact analysis on stale data.

Until recently, caching was considered to be the best closed solution to enable fast transactions and analytics, and companies typically preferred spending their hard-earned cash on those solutions. But these have bumped up against significant limitations of scale and reliability.

Today, however, in-memory HTAP solutions are becoming more common thanks to new developments in memory-based storage like SSD and Flash and databases that can leverage non-volatile memory. The simplified architecture of HTAP solutions also offers considerable cost savings potential compared with two side-by-side solutions — one for online transactional processing (OLTP) and one for online analytical processing (OLAP).

That’s all well and good, but what are the actual use case scenarios where HTAP comes into its own for IoT? The answer, perhaps unsurprisingly, is a large and growing number.

Posted on