In these waning days of summer it is common to reflect on how the days were passed. For me, it has been a wild few months. I spent these hot and sticky New York City days building the vision, team, and financing for a new company. I am incredibly excited, and honored, to report that last week was our first “official” week of work.
We are Alluvium, a team of engineers, data scientists, and executives building deeply integrated tools and services to deliver value for industries facing data challenges in the physical world. We believe that data emitted in the physical world is unruly and unrealized, but holds the keys to today’s largest business challenges.
Our mission is to build products that address these challenges, and to give the people working in these industries enhanced abilities. The journey has begun; but, how did we get here, and where are we going?
How did we get here?
The industry of data, or “big data” if you prefer, is young. But, not so young that it is without perspective. I have spent my whole career — now over a decade — working in data, of all sizes. The “big data” industry started in earnest around the mid-2000's with the development of a few seminal technologies that provided useful abstractions for both distributed data storage and computation. These technologies were developed primarily to support improved web search, and their historical origins had a large influence on how the ecosystem developed.
The technology was developed by and for products generating data in the digital world, and the first generation of the industry focused on building products to solve those problems.
An ecosystem of tools, services, and companies have been built to address these digital problems. This is by no means meant to downplay those contributions. A decade later, and we built some amazing technology and products. These are, for the most part, solved problems. What remains unsolved are data problems in the real, physical, world.
The next decade of the big data industry will be about solving these problems. Borrowing what we know about building highly available, scalable, smart systems, and inventing new systems for analyzing streams of data emitted when analog actions and decisions occur.
This is both a natural progression of the industry, but also a fundamental shift in the kinds of technologies, people, and companies that will constitute the next generation of the data industry.
Where are we going?
The promise of better living through connected devices, the so-called ``Internet of Things,’’ has captured the popular zeitgeist. While the entire consumer electronics industry may be set to instrument the lives of consumers — from wearables to smart homes — much of the focus remains on designing and marketing devices that engage consumers over a long period time.
All of the attention paid to imagining this whimsical future belies the present reality: there are massive industries emitting countless streams of data today that are remarkably underutilized.
The next frontier is not about making comfortable lives better through connected devices. It is about building data-driven products that make the hard, dangerous, and crucial jobs that power the global economy frictionless, safer, and more reliable.
In all data there is humanity. In every bit there are traces of of this humanity: in how a choice is made, or how a system is built. In the physical world the complexities of this humanity are magnified. To manage this complexity requires both deep technical expertise and innovative engineering. It also requires considerable empathy for the human beings behind that data.
Those of us who work with data are fond of describing it as messy, but data from the physical world is more than simply messy. It is knotted up in the perpetually flawed mechanism used to convert analog actions to digital signal, and the humanity that underlies it. The complexity of this humanity, however, is also our greatest strength and opportunity. The expertise, experience, and bias that people imprint on the data provide material for building great products.
This is where our journey begins.
Where are you?
We have started with a team that I am extremely proud to call my colleagues and partners. They have built some of the most used and recognized products in both consumer and enterprise data analytics. The combined expertise already under-the-hood here at Alluvium is intimidating in the best possible way.
But, hard problems require many more smart people to solve them.
We are in the very early days, but if this scale of opportunity and challenge is something that excites you, and makes you want to jump out of your chair and start building, we want to meet you.
It is time to solve the next generation of really hard problems.