Back to the Basics of Everything Data — Data Culture, Data Integration, and Snowflake’s Data Cloud
In the first of a two-part series, David Hrncir at Hashmap and Ajay Bidani at Powell Industries go back to the basics of everything data.
“We’re going to talk about data — everything data — so data integration, data technology expansion, some of the [really popular frameworks and applications] right now, all sandwiched between two levels. That’s data culture.”
— David Hrncir
Hashmap, an NTT DATA Company, recently traveled to fabulous Las Vegas to attend and sponsor the incredible Snowflake Summit 2022. Set amongst the backdrop of the bustling Las Vegas Strip, the 4-day event brought together some of the best minds in technology and data science.
Due to Hashmap’s demonstrated history with Snowflake as an Elite Snowflake partner and being longtime, passionate Snowflake enthusiasts, Hashmap is well-versed in Snowflake. We at Hashmap have a commitment to doing data better, together — especially when it comes to applying our knowledge of Snowflake when working with customers to find solutions to their data problems using the Snowflake Data Cloud.
So take Hashmap’s knowledge of Snowflake, and add the fact that our data podcast, Hashmap on Tap, has featured several guests from Snowflake, discussions involving its capabilities, impacts on the Data Cloud (and more!), and what’s the result?
You guessed it — this culmination of events presented the perfect opportunity for Hashmap to present a live episode of Hashmap on Tap at Snowflake Summit. We know it wouldn’t be Vegas without entertainment (and don’t forget the mimosas!)
So what was “on tap” for this session?
David Hrncir, Regional Technical Expert at Hashmap, and Ajay Bidani, Digital Enablement and Insights Manager at Powell Industries (and a Hashmap client), took the stage to present a Snowflake-centered, exclusive version of Hashmap on Tap.
To properly celebrate the occasion, David and Ajay popped the cork on a bottle of champagne to enjoy mimosas — a brunch favorite and definitely a fitting choice for an 11 AM session (if I don’t say so myself). While sipping on their mimosas, David and Ajay host a lively discussion about a wide variety of topics and challenges in the field of data science today.
In this article, I’ll walk you through some of the highlights of the topics that David and Ajay discussed on stage.
Data Culture
“We’re NTT, right? Literally, we live and breathe data culture. That’s really all we do. I can’t really even talk to you about it without involving data and data culture.”
— David Hrncir
Data Strategy
How would you respond if someone, right now, asked you the question, “What’s your data strategy?”
If your initial response is that of a deer in headlights, that’s not a good sign. On the other hand, if you can lay out 20 goals of your data strategy, then according to David, you’re on the right track, since one of the most critical pieces when it comes to working with clients is discovering their data strategy.
When asked about Powell’s data strategy and pushing towards modernization, Ajay places emphasis on the need to establish a foundation that consists of going back to the basics and keeping things simple. To make this happen, they are focused on keeping users engaged when it comes to data, acknowledging there is definitely a gap when it comes to self-service and scalable collaboration for people, and turning insights into action.
“It’s really about insights to action, but we’re trying to make things very tangible and speak more to the users about the time — the amount of time — they spend with data and making it optimal.
The hope is that by keeping it tangible, you can make it a part of the culture and really get people to want what happens next.
We’re all about thinking about change right now.”
— Ajay Bidani
Data Strategy vs. Data Culture
So how does data strategy fit in when talking about data culture, and how do you promote, discuss, or reinforce data culture within your organization? According to Ajay, “the conversation about data strategy is basically the exercise of trying to promote culture without using the word ‘culture.’” To encourage data culture, they are focused on increasing awareness and talking about the data they already have, and taking the insights gained from that existing data to determine what data they may need in the future.
“You can’t start an assembly [line] if those things aren’t there.”
— Ajay Bidani
Data Culture Starts With Leadership
When it comes to data culture, getting leadership involved is absolutely crucial in promoting, discussing, or reinforcing the topic and necessary behaviors or actions. If leadership is involved in this process, it’s ultimately going to help in attaining the organization’s goals of a strong data culture with a data-driven mindset.
David presents the results of a recent survey, in which the number of companies referring to themselves as data-driven has gone down from 37% to 31%. Additionally, 60% of all data analytics projects fail, 79% of all data projects had too many errors to be trustworthy, and 87% of all data science projects never make it to production.”
David found these results to be particularly shocking— primarily because of the decrease from 37% to 31% of companies that described themselves as data-driven. In his opinion, this shift is a reversal of what we think of as being the standard in this industry.
Why A Data-Driven Mindset Matters
From Ajay’s perspective, without a data-driven mindset being pushed by leadership, failed projects will seemingly always continue, but they’re never going to get any higher. These projects may get worse and worse over time when it comes to the challenges you’ve had or investments that you’ve made. As a result, this can cause your systems to become suboptimal and you may end up in a situation where the user experience starts to suffer. The moment the user experience starts to suffer, then you lose your ability to do more because there’s not a lot of trust in what you’ve been doing.
This leads Ajay to bring up another point — “When operating lean, you forget what it’s like to have people in the right roles as opposed to wearing multiple hats.” How would a situation like this affect an organization’s data culture? It certainly says a lot about the data culture if you can’t “try and take those hats off” as David suggests. It’s important to recognize that there’s value in making it a necessity to be focused on people’s accomplishments and goals, and developing this mindset is incredibly important to help organizations change for the better.
“Organizational success is going to come down to leadership promoting this data culture.“
— David Hrncir
Data Integration
It’s impossible to attend Snowflake Summit and not talk about data integration. Many leaders in all types of organizations are either looking into data integrations or will need data integration eventually. David states that “I don’t like to use ETL, ELT, ETLT, RELT — it’s just too complex to say all these different acronyms so I say DI [Data Integration] — it’s the easiest way to do it.”
What’s with all the acronyms?
So what is Ajay’s perspective on organization ETL and how does Powell Industries look at it? How does he prefer to work with those different acronyms, and what do they mean to him and his organization?
“What’s unfortunate about it is that we have fallen into the state of mind that ‘this’ means ‘this’ because of how we experience it today.”
— Ajay Bidani
According to Ajay, “I would be lying if I said that acronyms are not our best friend, and to be honest with you, we do it way too much — just like everyone else.” He continues by saying, “We definitely talk about (and use) the term ETL and ELT regularly. However, what’s unfortunate about it, in his opinion, is that it can cause you to fall into the state of mind that ‘this’ means ‘this’ because of how we experience it today.” So from his perspective, acronyms are helpful, but they shouldn’t be static — it’s important for them to continue to evolve over time.
What are the roles of data ingestion and data transformation?
When it comes to data integration, it covers a couple of topics, such as ingestion and transformation. At Powell, Ajay shares that they have primarily been talking about those concepts individually because their focus has been on what they’ve built so far and the exact thing they’re trying to do.
However, when you’re thinking about DI, you shouldn’t be thinking of it solely as an ELT tool. When working on projects, one of the most common requests is DI for reference architecture. David states, “We are trying to build these reference architectures and playbooks, ultimately, to have more successful future implementations.”
Reference Architectures and Playbooks
So can reference architecture and playbooks be used as “cookie cutters” that work in your organization? If you want Ajay’s honest answer, he says, “Not especially.”
Reference architectures serve as a good point of reference, especially for learning, but he personally prefers to take reference architecture as “a place for interrogation.”
He continues by saying, “It’s kind of like saying, ‘These are pieces. Why are these pieces important?’ In my organization, there’s no way that we’re going to do something because someone came in and said, ‘Hey, this is the thing to do.’” Ajay’s statement is definitely an interesting way to think about reference architectures and let the organization put things into perspective.
The first question an organization is going to ask is, “What does it cost?” The organization needs to understand the end result and the ways that it fits into the reference architecture in order to justify what you’re trying to do. Ajay believes that the reference architecture serves as an internal way to help yourself understand what is necessary to reach the desired end result. As Ajay concisely puts it, “Well, it’s reference architecture. It doesn’t tell you everything. It tells you the kinds of things you need.”
Then, the challenge lies in trying to understand those foundational lessons better as well as how they affect what an end-user would actually get experience-wise. Once you have a better understanding of exactly how those play into that experience, you’ll find that building experimentation into strategy — and doing it often — is obviously the key.
Experimentation tends to bring with it a new slew of buzzwords and acronyms. While these may just be acronyms, it’s important to think about the question, “How can these meanings and uses differ within the organization?”
Some of the acronyms David discusses include:
- Proof of Concept (PoC) — the architecture as a whole; doing things that have never been done before to achieve your end result: a good (working) piece of software architecture
- Proof of Technology (PoT) — investigating a software vendor or a service to see if their technology is capable of performing the functions it claims to be able to do
- Proof of Value (PoV) — understanding how something works when put into practice; asking “Do I get any value from this?” to determine if its benefits (like the fact that it saves time, uses less manpower, etc.) outweigh the monetary cost
- Proof of Purchase (PoP) — the ability to meet a deadline and a cost for a project; determining whether or not the cost for a project can actually be met by iterating the process
Ajay brings up the example of a minimum viable product. His philosophy is that instead of aiming for a minimum viable product (MVP), take it and add complexity to it. Look to see what future iterations could include and what it will take to get from one step to the next. These actions can help to set yourself up for success so you don’t let yourself get trapped within the MVP. Additionally, by involving your organization in adopting this process, it will allow increased support for getting from one step to another.
Prioritizing and Simplifying Data
How can a minimalist mindset be used to make this concept less complex and remove some of the moving parts? David uses the analogy of a house to break this question down. Most houses aren’t carved from a giant rock — they are formed by building small parts incrementally. In other words, you have to lay the foundation first. If you try to go from one brick to a mansion and hope it works, that is essentially a recipe for disaster.
“I’m trying to get the business to buy into that way of doing things versus telling them what they need. . . [The] amount of data is the easy way to tackle that.”
— Ajay Bidani
When it comes to making things simpler with tools, it’s important to look at the amount of code that’s involved — having to learn a front-end tool that requires a lot of learning could be more complex than other available tools. In terms of cost models, Ajay says that “Less is more . . Think about how many different levers do you have to pull?” It’s easiest for him to think about this concept by quantifying the amount of data as opposed to how much it’s hit (or how much compute for how many times in a day).
By quantifying the amount of data, it can make this process easier and allow the potential for planning. He goes on to say that, “How many times I hit it is a little bit more like trying to sell lean — or why I should need it less.” This conveys that it’s imperative to identify the requirements and goals of the data to ensure that business decisions are made in alignment with those factors. In David’s experience, clients that have the most success are the ones whose leadership already has a strong data mindset.
“Success is whether or not we can use our agility to improve and be successful.”
— David Hrncir
Wrap-Up
What happens in Vegas stays in Vegas, so we can’t give you all the details—you had to be there to experience it! — but I hope this article helps you in understanding some of the most pressing and interesting topics in data today. Did you go to Snowflake Summit? If so, what was your favorite session and your biggest takeaway? Let us know in the comments!
Look out for our follow-up next week where David and Ajay will cover a wide variety of topics on technology expansion, challenges in the modern data cloud, data warehouses, and more! In the meantime, if you want to hear more from David and Ajay, you can read their info below and check out links to listen to their previous appearances on the Hashmap on Tap podcast.
Finally, learn more about how Hashmap and Snowflake can help you do data better, together here.
About the Hosts
David Hrncir — Regional Technical Expert at Hashmap, an NTT DATA Company
David is a Data Cloud Expert at Hashmap, an NTT DATA Company, and is an enterprise data technologist and Snowflake enthusiast with an affinity for helping organizations build their data strategy roadmap. He’s been in the application development and data engineering game for over 27 years and has experience in numerous verticals.
Listen to David on these episodes of Hashmap on Tap:
- Episode #70: The Facebook Insights Show Featuring Fivetran, Snowflake, & Looker
- Episode #71: The Updated Snowflake Estimator with Sam and David
- Episode #80: The Cloud Data Platform Benchmarking Show with Chinmayee, David, and George
- Episode #83: The Data Ops Show with Chinmayee and David
- Episode #88: Leaning In On Semi-Structured Data with Chinmayee and David
- Episode #100: Hashmap on Tap 2021 Flight: A Sampling of Our Favorite Podcast Moments
Ajay Bidani — Digital Enablement and Insights Manager at Powell Industries
Ajay is a seasoned data veteran performing technical alignment for Powell working with numerous business functions to provide time-saving automation and optimization with a focus on creating scalable architectures. He has an affinity for driving business value through data stack and business application roadmap development.
Listen to Ajay on these episodes of Hashmap on Tap:
- Episode #29: Leadership Perspectives with Ajay Bidani from Powell Industries
- Episode #68: Manufacturing & Data with Ajay Bidani at Powell Industries
- Episode #97: Data Integration, Fivetran, and HVR at Powell Industries with Ajay Bidani
Listen to Hashmap on Tap’s most recent episode:
🚨 Subscribe to Hashmap on Tap on Spotify, Apple Podcasts, & Google Podcasts. 🚨
Additional Resources:
Ready to Accelerate Your Digital Transformation?
At Hashmap, an NTT DATA Company, we work with our clients to build better, together. We are partnering with companies across a diverse range of industries to solve the toughest data challenges and design and build data products — we can help you shorten time to value!
We offer a range of enablement workshops and assessment services, data modernization and migration services, and consulting service packages for building new data products as part of our service offerings. We would be glad to work through your specific requirements. Connect with us here.
Holly Hilton is a Marketing Associate at Hashmap, an NTT DATA Company. She writes blogs (and other content) for Hashmap and is a co-producer for the Hashmap on Tap Podcast. When she’s not busy crafting content, she can most likely be found with a YA novel or Xbox controller in her hand, ready for the next adventure.
