Improving Organization and Access on Civis Platform

Navin Gopaul

The Civis Platform is the foundation of the solutions that Civis provides to clients. This fast and powerful tool enables organizations to make informed data-driven decisions with confidence. A major effort in 2018 for the Civis tech team is a rewrite of the platform UI using modern JavaScript technologies. For my internship this summer I was tasked with supporting this project, particularly through re-implementing the data pane in the revamped platform UI. The data pane is the primary tool that allows platform users to navigate schemas and tables in their organization’s Redshift cluster.

Civis Platform with the old data pane

The old data pane appeared beside the project pane and provided users:

  • A list of schemas on their cluster
  • A dropdown with tables within those schemas
  • A dropdown to reveal surface level table details
  • An option to import new data
  • The ability to search schemas

This old pane faced several usability issues. The overall placement of the data pane was problematic. Multiple user interviews with internal data scientists at Civis revealed that having both the projects pane and the data pane open at the same time unnecessarily cluttered the interface of platform.

There were also issues with the functionality of the data pane itself which can be best understood through the lens of Norman’s Ten Usability Heuristics. The first heuristic we’ll look at is the visibility of system status which is the idea that “systems should always keep users informed about what is going on, through appropriate feedback within reasonable time.” The elements of the old data pane that violate this heuristic involve faulty behavior of loading at the level of schemas and tables. Opening the pane would often just display the name of the cluster with no indication that schemas are in the process of loading. The issue of loading at the tables level was problematic in a slightly different way. Upon clicking on one of the schemas, a loading animation would instantly appear but persist longer than necessary. The list of tables would appear and the loading icon would remain above the list of tables, leaving the user confused as to whether or not they should wait for more content to load.

In addition to visibility of system status, the old data pane had problems with the consistency and standards heuristic. This is the idea that elements in an interface that appear to be similar should have similar functionality. In our old data pane, clicking the name of each schema drop-down would reveal the list of tables contained in that schema. However, clicking on the name of the identical table dropdown would trigger a refresh of the current view on Civis Platform, bringing the user to an entirely different table details page rather than functioning as a dropdown.

This brings us to user control and freedom. This principle states that there must be a clearly marked emergency exit, allowing users to leave an unwanted state. Using our old data pane, users would not be able to retreat to their previous spot in the data pane through the table dropdown. Rather, users would have to reopen the pane and manually navigate to the previous state.

The redesign and implementation of the new data pane sought to address these usability issues as well as add functionality that would allow users to quickly access information about their data that they need most. The design phase began with diagnosing the problems with the old data pane and then designing and iterating on high-fidelity prototypes. This iterative process involved working closely with Civis’s design team to conduct interviews with in-house data scientists who are frequent users of Civis Platform. In these interviews, we conducted robust testing of our prototypes by running through the flow of functionality with each user, identifying pain points, and updating the prototypes accordingly. These interviews also involved asking users about their workflow with the existing data pane, in order to identify workarounds that we could devise solutions to. Throughout this process, there was discussion among engineers and designers to assess the technical feasibility of the proposed ideas. After several iterations of this process, the design was finalized.

The new data pane integrated with the projects pane — the state on each tab persists when the other is toggled

In the final design, we decided to integrate the new data pane as a tab on the left sidebar which originally only had information about the organization’s projects. This design choice saves a great deal of space on the main page of platform. If the user wants to quickly access information on both of these panes, the state of each tab persists whenever the other one is toggled.

Loading behavior of the old data pane (left) vs. the new one (right)

On the new data pane itself, schemas and tables are rendered in a drop-down manner. Clicking on the header reveals the list of tables contained within that schema with a clear indicator of system status via a loading modal overlay appearing and promptly disappearing when tables are successfully fetched from the server.

Search functionality of the old data pane (left) vs. the new one (right)

Additionally, this new view also provides the user the ability to sort their schemas alphabetically and reverse alphabetically at the schema and table level. The search behavior was also refactored in this new design. Rather than appearing above the list of schemas, matching search results are integrated into the list of schemas/tables so the user can easily visualize the hierarchy of their data.

The table details view that appears when a table in the schemas drop down is clicked on

The final part of this re-implementation is the table details view that appears when a user clicks on an individual table. We decided to include basic information such as the table/schema names and the description of the table. The user interview process allowed us to identify what details are most important for users to be able to access quickly beyond basic table information. We decided to include:

  • The owner of the table
  • The last time the table was refreshed
  • The number of rows/columns
  • The size of the table
  • A list of all of the columns in the table with labels indicating which columns are distribution and sort keys

We also discovered that users were frequently copying and pasting column names and manually separating them with commas, so we provided a clipboard tool to instantly copy the table’s column names as a comma-separated list. Lastly, a “View Details” link is provided which takes the user to a view on the main page with in-depth information about the table.

Overall, the process of redesigning the data pane in Civis Platform highlights the importance of garnering feedback from your users along every step of the iterative design process in order to provide the optimal level of value in the final product. Following internal beta testing, the redesigned data pane has been met with positive reception from Civis’s in-house data scientists. Our in-house data scientists have noted that our redesigned data pane is more responsive than the old one and makes finding information about tables more accessible. The redesign has resolved many of the usability issues faced by the old pane, enabling platform users to seamlessly organize and access their data, therefore allowing them to focus on what’s important — leveraging the powerful tools provided by Civis Platform to produce informed, data-driven decisions.