Dashboards, a data scientist best friend
Data science consulting is a hard business. The first few weeks and months are a mix of sales and marketing work. Many times, a thrilled customer can just disappear when they discover the costs, or for other different reasons that are not even related to us. So what makes a data science project kickoff well and succeed?
Well, many things, of course, but one crucial part is communication and stakeholders’ engagement.
Making the stakeholder engaged fast with proven value is key.
Data scientists work very hard on their projects, they feel excited when they find an insight, and they want to engage the customers or managers and get them excited as well. They usually do so with a PowerPoint presentation or the likes. Well, slides can be boring. We have all seen too many of those. So, what can we do?
Engagement is enhanced with interaction and a story, so we need a naturally interactive tool that tells a good story, something clickable that works. One way to do just that is with a dashboard
Dashboards are an extremely efficient way to involve the customers in the data science project. They can serve as an online and accessible way to do: Demos, Tools, MVP, or even an actual product, if connected to some database, or any other dynamic way to refresh the data. But another great thing about it in the data science lifecycle is that they can also show progress during all those times that nothing major is yet to happen. Sometimes just showing the customer an interesting interactive view of their data is exceptional.
There are some nice-looking tools for dashboards that cost money, like Tableau. But it is not only their cost that limits us. These tools are either restrictive in nature, require a lot of work and/or lack many capabilities. While using python, the data scientist is free to independently use whatever they already know and need.
Some of the most free and successful dashboards in the python world (the DS main programming language) are Dash, Panel, Streamlit, Voila. For a fuller comparison between these libraries, have a look here.
Dash and Streamlit work using python scripts while Voila and Panel can work directly from a Jupyter notebook using widgets. This last capability can be very attractive for data scientists to showcase their work and engage customers without spending too much time on development. Voila will probably be the simplest and fastest way for a data scientist to create a dashboard nowadays since it’s just rendering the Jupyter notebook almost as is.
In the following, I will share some experience we had in our team and a super simple implementation for a Voila server that gives the look and feel of a web-app.
Before jumping to Voila, I will just say that our DS team uses Dash and Kubernetes to build very nice-looking dashboards, which can also be complex on the backend. The trouble is that the open-source Dash is too complex for a data scientist to quickly deploy a dashboard that brings value. It also only uses Plotly visualization, which can slow deployment even further if one needs to change already developed visualizations. Still, Dash is an excellent way to build products, tools, and demos.
So, we have explained the need for a simple and fast tool, and we also found it in Voila. What do we do now?
We can deploy a voila server on a dedicated Kubernetes pod per app. Here is an example code on GitHub. Let’s have quick look at a simple app.
Running the server
The above code is used as the starting script on the pod. After the pod is being deployed on the Kubernetes environment, it runs the above script to install whatever libraries it needs that may be missing on the docker image or need updates. Later it uses a chosen port and Jupyter notebook to start the server. Exposure and/or mapping of the ports is done using the Kubernetes or Docker tools. Notice that we can always have a dashboard ready for a specific data scientist to just come and replace or update using this methodology. So, after the first deployment, we may not even need to deploy more pods unless we need the former app to stay in place.
For those who are not restricted by security, it is also possible to deploy Voila using mybinder and other services.
A simple app
The above is a Voila dashboard that has a look and feel of a web app. Let’s view the simple code that created this.
Notice above that any markdown text will be shown in the app, therefore we do not use markdown for chapter headers. Instead, we use comments (#). But markdown works great to highlight text in the app.
In the above, we commonly load data. Declaring a widget and an interactive function that will use its result after the user will interact with it — more on widgets.
We generate the dropdown list and plot in a very similar fashion to what we have seen before.
Summary
We want to quickly engage our customers with the value of analyzing their data. Web-based clickable dashboards are a great way to engage them and provide value fast. Voila is a great solution for deploying a dashboard with minimal effort and maximal value. We went over a simple use case and code to achieve the experience of a web app with Voila.
