The future runs on NoSQL because the alternative is chaos
I’m writing this piece after attending last week’s MongoDB World 2022 in NYC.
MongoDB is the most widely-used NoSQL database and the major alternative to a traditional SQL database. I attended the annual conference as part of the DEI Scholars Program.
Here’s a photo of the DEI panel where scholars had the opportunity to share our experience at the conference, our journeys to tech and resources that helped us overcome barriers in the industry.
I’ve included a list of the organizations mentioned during the panel at the end of this article.
MongoDB empowers underrepresented people in tech (like women and people of color) to attend the conference as part of the program. The experience was extremely impactful for me since I had the chance to meet and learn from leaders across the world who are shaping the future of databases.
In this article, I’ll explore the ways traditional databases don’t meet the world’s rapidly shifting data needs and consider the real world implications for people depending on legacy systems.
I assume some SQL/database knowledge but aim to make this information accessible to readers at all levels and across various sectors.
I hope it inspires underrepresented people to consider STEM and starts conversations about how databases can be reimagined to support better outcomes.
From large banks to government agencies to nonprofits, data is being created, manipulated and stored across sectors like never before.
As leaders look to improve their software applications and buy tools to meet new and growing business needs, they’re deciding on the future of their data and whether or not they will be able to solve the problems of tomorrow.
This means that the future of the people using those systems (and the users they support) is also directly shaped by the database being used.
The wrong database model means that organizations have no way to make sense of their data.
And, for the purposes of this piece, that’s a traditional relational database management system (RDMS) that runs on SQL which, according to TechRebublic, makes up 72.8% of all databases.
Reality #1: Data is getting more complex
Throughout the COVID pandemic, health service providers had to quickly respond to the public’s needs such as recording breakthrough rates for patients. Legacy systems are rigid and prevent organizations from meeting new and growing needs.
Data is becoming more complex — databases need to keep up.
For example, organizations are capturing more time series data which requires specialized approaches to analysis. However, when these fields are added to new or existing systems, creating the connection between the application and the backend database is time-consuming. In a SQL database, the developer has to update a rigid, stale schema (the underlying structure of the database) each time a new field is added to an application.
Compared to SQL, NoSQL databases allow users to create documents (records) without a uniform schema. This is more realistic since teams can make updates to business applications as the needs of users change.
Reality #2: Real world data is messy
How do legacy databases store and serve queries for direct service providers that need access to data that is commonly missing, such as for survivors who report sexual assault?
They don’t.
When data comes from different sources, relational databases don’t provide enough flexibility to allow data that is missing from one source to be found in another.
It’s nice to think that every organization has the bandwidth to structure (and restructure) data to accurately store business application data.
This is not realistic.
Most organizations, especially those running on legacy SQL systems, don’t have enough technical staff or funding to build these schemas.
This has major consequences for the people those databases serve.
NoSQL databases solve this is by providing developers with a complete schema (an overview of the structure for multiple collections/groups of data). Access to a schema means developers and data practitioners gain a full list of the fields available across multiple sources — and allows them to form better insights based on that data.
Reality #3: Relational models hinder creative problem solving
When organizations are too committed to the old ways of doing things, they might only hire people who remind them of those they’ve hired in the past. This means overlooking the talent and potential of people of color, women and other underrepresented groups.
Trying to structure data and solve real world problems using a traditional relational approach doesn’t allow organizations to innovate.
When creating abstractions that depend too much on how the problems have been solved in the past, people can’t create new connections or view problems in creative ways.
This is both a technical and psychological barrier to innovation.
One of the underlying qualities of relational databases is that tables must have predefined relationships through primary and referencing keys. NoSQL eliminates the need for these prior connections since collections of documents can exist in the same databases without the need to predefine the relationships between them.
Conclusion
I hope this article provided a background on the limitations of SQL databases, the real world implications of these models and ways NoSQL addresses these gaps.
As the world’s data needs evolve, data practitioners will consider how databases can scale and adapt to fit the rapidly changing landscape. These conversations would benefit from including the perspective of people who understand the real world implications of data structures and systems.
The world as we know it still runs on SQL. If and when organizations choose to transition to NoSQL, building out new databases that meet the needs of users won’t happen overnight.
Learning about SQL and NoSQL query syntax, different data types and summary statistics is a good way for people new to the field to get started.
Resources for underrepresented people in STEM
Brilliant Black Minds — an organization that provides access to practice interviews to help aspiring Black engineers successfully enter the tech industry and change the world.
Color Stack — Helping Black and Latinx Computer Science students get degreed and hired nationwide.
Women in Data — Supports opportunities in data for women throughout their career.
Women Who Code — an international nonprofit dedicated to inspiring women to excel in technology careers.
