What we learned at Future Data 2020
Building scalable sources of truth, staying fast and flexible, keeping humans in the loop, and more.
It’s been one week since the first Future Data, and we’re still soaking in everything we learned. Last week, over 700 people at the intersection of data, systems, engineering, and analytics shared a fascinating look into what’s next in our industry.
There are too many great moments to highlight, but we wanted to take a step back and look at some of the insights this new community shared on where they think the world of data is heading. Here are just a few of the threads we noticed:
The Future of Data Needs a Human in the Loop
While much has been written on how Machine Learning and automation will reduce the need for human eyes on data, many of our speakers rejected this idea. As they discussed, to make the right decisions and provide oversight on algorithms, the future of data can only be achieved by keeping humans in the loop.
In his keynote kicking off the event, Pat Hanrahan asked, “Will AI make it unnecessary to interact with data? What is the role of humans in analysis over time? Do we need people to be involved at all?” He argued the answer is yes, explaining that we need people involved because of the “nature of uncertainty, the nature of the decisions, and the nature of our responsibility.” He argued, “we could just turn it all over to the computers, but I don’t believe that’s the right way to go.”
Building off this idea of uncertainty and context, Azza Abouzied argued that the goal of analytics is to provide recommendations for next actions and allow humans to make the decisions from there. As she explained, “The crux of data analytics is to deliver insights from any data, big or small. The purpose of insights is action, and decision making is choosing the right action from a number of options.”
While AI and advances in Machine Learning will make it easier to understand uncertainty, get important context, and help automate exploration, the future of data will require analysts, decision-makers, and business operators to use these tools to make informed decisions and test the validity of their recommendations.
Building Scalable Sources of Truth is Critical
To bring about the future of data, several speakers explored the tools and data stack necessary to analyze our growing data sources fast enough to make critical decisions.
Tristan Handy walked through a bit of history and how the last four years have enabled the Modern Data Stack. He explained, “The technological underpinnings have been rewritten and shored up, making a foundation that is now ready to have the next-gen of technology built on top of it.” He argued that the Modern Data Stack will depend on “tools to manage growing data assets, allowing self-service to be realized, new vertical-specific tools used inside your stack, and data pipelines flowing in near real-time.” To make effective decisions, your business teams rely on real time, current data, and Tristan thinks the coming changes to the Modern Data Stack will allow new uses and integrations for the future of data.
Matei Zaharia also brought his unique point of view on this emerging stack, specifically exploring how we store our growing volumes of data. He asked, “What if we could get all the capabilities in cloud storage?” and proposed adopting Data Lakehouse technologies, which could make enterprise data timelier and more reliable.
Finally, to have that reliable source of truth, Barr Moses argued we need to address the problem of “data-downtime” — those periods where data is partial, erroneous, missing, or otherwise inaccurate. She explained, “Data downtime is the biggest problem you’re not thinking about — costing millions of dollars and engineering resources.” The key to addressing this problem is by being proactive with data observability saying, “To allow for the future of data-driven decision-making, we have to get the fundamentals right.”
Ultimately, to quickly understand data as it changes, our speakers argued the Modern Data stack will need to be reliable, fast, and trusted.
The Future Must be Fast and Flexible
With more data than ever before, the key to unlocking what’s next in data lies in flexible tools that allow for fast, accurate analysis and real-time decision making. Scott Penberthy put this succinctly, saying, “Analytics, with incredible power, on demand is the new toolset we’re going to use in the future.”
Ben Horowitz discussed this need in terms of building a data-driven culture and using data to make critical product and business decisions. As he explained, to truly be data-driven, you need tools that will allow you to find answers as quickly as you ask the questions saying, “You need to be able to ask the questions, get a fast and direct answer, and then you can have a data-driven culture.”
This level of speed is critical in staying competitive, Ben cautioned, saying, “With so much data for decisions, if you’re doing everything by intuition in like 2025, it will be impossible to compete with any company that knows how to use data to make sharper decisions.”
The Future of Data is Accessible to All — With a Little Help
Throughout the event, as speakers explored topics from the impact of data in elections to the ability of new data tooling to empower business owners, many speakers came back to the same idea that to truly explore what’s next in data, we need to think of analytics as a team sport. Alan Jacobson said it best, saying, “If you have a mind, you’re an analyst.”
To unlock informed and contextualized decision making, analytics needs to be more accessible to business partners and analysts need to be more ingrained with the business. “Operators know where the gold is buried, if you teach them a bit of analytics they start seeing opportunities and can either get answers themselves or tell analytics how to help them get it,” Alan Jacobson explained.
Getting to this ideal of accessible data and analysis requires data professionals to enable their business partners to feel more comfortable in analytics and to open the community to everyone through our tech. As Alan explained, “As data science professionals, part of our role, is to open the community to everyone through our tech, by investing in training, and by coming up with ways to make this the most diverse and inclusive environment that we possibly can.” Scott Penberthy backed up this challenge saying, “What we need to do as a community is to make what you do as analysts as easier. So that people outside of us, the business users, can do analytics as easy as making pizza. They don’t need the whole pie, just a slice.”
The Future of Democracy is Data
When it comes to data and democracy, Elena Grewal took it one step further, encouraging the data and analytics community to make understanding data more accessible to all saying, “The future of data is to help the public good. While the truth should be self-evident, it’s often not. Data can be helpful to understand what’s actually happening. There are so many stories using data to tell stories that are not truthful. But this is our moment to really dig into the data and figure out what’s going on and to use data to help the public good.”
Whether it’s within a company or to make the truth evident in a democracy, the future of data requires the data science community to work closely with experts outside their field. This will equip business partners to better understand the data and provide data specialists with the insights they need to find answers in the data.
These are just a few of the themes we saw throughout the event, but what was clear in all the talks is that the future of data is collective and collaborative. As Peter Bailis said, kicking off the event, “Future Data isn’t just about one piece of analytics — it’s about everything in data and analytics.”
What were your takeaways from Future Data? We’d love to hear your thoughts, if you attended the conference, fill out the survey in your email with your feedback. If you have more to say, or think an idea wasn’t covered at Future Data 2020, write it up and submit it to this publication. We’d love to share your ideas!
P.S. — To catch up on any topics you may have missed, you can now view every session on-demand.
