Managing Data with Design
Reimagining data management UX with natural language to build Ship
Data management is hard.
That goes double for startups & small business that don’t often have the time to sink into specialized internal applications, and then find all the possibilities of their data limited to the few on the technical team.
We think design can help change that, but how do you demistify the database, and make it useful for everyone?
The idea behind Ship is to simplify data management and analytics for startups & small business, letting anyone at a firm use & learn from their data. With UX at its heart, the first step to solving that problem was taking a closer look at how they work now.
How They Work Now
To start with that, we visited a few. Digging into it, different startups & small businesses use everything from Excel to ZOHO Creator to mySQL—and everything in-between. The range from simple spreadsheets to complex databases seemed to depend on how influential the technical staff were, or whether there were any at all.
Accessing data on-the-go was a common pain point across solutions, as was a lack of flexibility in simpler implementations (like Excel) and a lack of accessibility in more technical implementations (like mySQL). This led to two goals with Ship:
- A platform that’s smart. Going in, we knew simplification would go far in making this useful to everyone, but it couldn’t be at the cost of functionality & flexibility. In fact, we aimed for the opposite: keeping the system simple while retaining the kind of advanced cases requiring computers in the first place. This meant building it smart.
- A platform that’s accessible. There’s a lot that can be gleaned & learned from a firm’s data, and it shouldn’t require a separate front-end or a deep understanding of SQL to get it. A platform where editing and reviewing data is as simple as as in Excel, but with powerful relational and analysis tools.
The approach of accessibility is taken a bit further as well, leading us to craft Ship as a platform that could be built on a Ship-owned cloud, or atop an existing database.
Flexibility & Access for Everyone
So how can all this be done?
The pivotal problem was the interface. It was clear that an SQL-esque CLI was not the way to go. At the other end of the spectrum, Excel is backed with decades of familiarity for even the less technically-inclined.
A series of spreadsheets it is.
Taking after Excel, however, had its own range of problems. Data I/O is simple enough, but how does this help users understand their data without reaching back to hand-configured charts and bar graphs? Simply put, it doesn’t.
The Search for Something More
To better grasp how users would want to use their data, we had to look beyond the data itself, to what they want to get from it. Users didn’t open the app to browse through collections of data to hunt and peck for what they were looking for. They wanted to ask questions.
What inventory items sold over 5000 units last week?
A simple enough question for another person—but for a data platform, this question meant pulling up a collection or two and messing with sorting and filters. So we decided to get rid of that.
Our early concepts led us to the “ad lib” approach, where users could fill in blanks that would ultimately give the same results as their question. The question would then become something like “Show me Inventory that matches Transactions > 5000 ordered by Number of Transactions.” There are worse approaches, but it had a glaring issue.
It wasn’t how users thought.
Not only did it force users to reconstruct their thoughts a certain way, but it constrained us from working with data analytics — “Chart sales of item A over the past week” — and data reporting automation — “Email me that chart every Monday morning” — not constraints we were willing to work with if we were going to let more users do more with their data.
This led us to take the concept of the question a bit more literally, using natural language processing (NLP). Popularized amongst users by Siri & Google Now, and in products from the notable Wolfram Alpha & Facebook Graph Search to the up-and-coming Clara by Clara Labs, NLP would allow users to ask questions about their data the way they thought about them. NLP is what we were looking for.
A Hybrid Approach
Utilizing NLP was a breakthrough—but not one without its shortcomings. Users weren’t always clear how the system interpreted the data before pushing results, and while finding and reporting data with NLP was great, most other actions became a chore.
To this effect, Ship became a more hybrid system. Using NLP with an option for a more traditional filesystem folder view to browse Collections of data made searching data a breeze and flexible to the user’s preferences. Once processed, the NLP query was broken down into a clear view of how the system understood it (especially important for automation), harkening back to the two-way understanding from the ad lib approach. And a simple, spreadsheet style view makes an appearance when viewing query results or a collection (alongside auto-generated charts and graphs for reporting queries).
A platform that’s smart. A platform that’s accessible.