Iridium: Embedded Computational Notebooks in Power BI & Tableau
👩💻 Data scientists everywhere have adopted 📓 computational notebooks like Jupyter and Observable to build and share powerful data models and visualizations more effectively and efficiently than ever before. Compared to the traditional software development workflow a notebook is a productivity powerhouse that allows rapid iteration. However, this power seems to be locked to the medium of hosted notebooks widely. Read more to discover Iridium, an embeddable notebook editor that works inside BI tools to get the best of both worlds.
On the green pastures of easy to use Business Intelligence tools such as Power BI and Tableau, users enjoy very powerful features to:
✅ Connect to multiple data sources (Databases, files, APIs etc.,)
✅ Create complex data models (Joins, relationships etc.,)
✅ Dashboards with filters, facets and cross highlight on interactivity
✅ Visualize data in standard charts (In built: Bars, Lines, Treemaps etc.,)
✅ Visualize data using custom charts (extensions and plugins)
❌ Iteratively analyze and visualize data/insights in an embedded notebook widget
✅ Export, publish and share to various channels
Iridium, is that missing bridge, the embeddable notebook interface inside your BI pipeline.
The Example: The gif you see on the top of this article is me tinkering with my strava runs, I imported the data into powerbi and it contains polylines which to powerbi are just strings and there is no easy way to put them on a map . I did not want to jump from one app to the other while analyzing my data, I also wanted to use turfjs to cluster my bike rides based on locations and color code them on a d3 canvas etc., I did all that using Iridium without ever having to leave power bi. Once these are built, its also fully integrated with the dashboard so I have cross filtering etc., out of the box.
Ok, how do I use it myself? If you like to directly download and try it for yourself, jump to the git repo and download it. Check the demo pbix for examples. Once you add the visual to your dashboard, drag the required fields, these will be available to the runtime as a variable dataView which can be used to create any web based visual or UI: such as but not limited to D3 charts, maps, tables, forms, interactive web apps etc., Or if you are a Tableau user, download the trex file and add to your dashboard.
The story of why it’s created: (tl;dr: bespoke data visualization development is a bottleneck in the BI pipeline) Being a data visualization developer and BI Consultant for several years, I noticed multiple camps of data visualization that thrived independently across the ecosystem, the process of creating a bespoke visualization piece went from niche to common commodity in many aspects and got much simpler too, but the bespoke part is still a puzzle without certain skills, around a couple of years ago I created a power BI custom visual that was submitted to the Microsoft PowerBI visual store as a free visual, It is an Animated Bar Chart Race visual that is quite easy to use. It was downloaded a few million times, got a decent rating and I was glad to see videos on YouTube of people reviewing it. It’s a simple D3 visual that was built over a weekend, and another week researching the custom visual API, the build environment and most importantly the process to submit it to the store (this part is optional, but you did not go this far to go just this far, so, I did, and for that I needed to pay and enroll into the developer program, prepare lots of demos, guides and policy documentation. I even had to create a website and a work email and wait for the review process to be completed before I was able to submit the visual to the store.) On the technical side, a visual is a packaged code running as an iframe on the report canvas, the process to build it requires writing typescript code, setting up some boilerplate and a nodejs based toolchain along with a decent understanding of the visual’s lifecycle: how it gets mounted, gets data, settings, bookmarks, filters etc., to test the visual requires signing local certificates and enabling developer modes and a few other steps, also you will have to use d3 and other libraries to build your visual of course.
Why to consider an alternative at all? (tl;dr: Data context not code context & notebook not IDE) It’s a good time to be a custom visual developer, there is demand for bespoke visuals and very few skilled designers and developers. Several people who used the visuals request minor enhancements, often little changes that would make it more usable in their specific dashboard or context. I know its a few tweaks here and there, but checking out that code, setting up the build and data stack, developing, testing, packaging, and resubmitting to the store and dealing with compatibility with previous versions, generating usage guides and demos was all a lot of effort for a hobby project. Yet, at the same time, I feel its not in the best interest of the community to monetize from building these visuals with their tiny variations or hard code only a few features as options for the end users (who are actually report developers not the end users of a dashboard) and not have flexibility to tweak things without having to go through another sprint of development.
This is getting too long! What's the solution?! What would benefit the overall community? How can developers spend less time with boilerplate, code and get to the data analysis and insights quickly? What’s the best medium to define graphics? These are very challenging questions that leaders in the field such as Mike Bostock have tried to address in their body of work, the result is https://observablehq.com/ which is a computational notebook medium with a collaborative cloud based online editor powered by its diverse community, I think of it as the data visualization silk road! The best part is that the observable team has open sourced the runtime and parser. Thanks to another awesome developer Alex, who created compiler library that converts observable markup to the runtime code, with all the pieces now available it was quite easy to build a minimal no bells and whistles embeddable version of this notebook interface using preact, codejar and javascript. The editor is available on GitHub here: https://wishyoulization.github.io/iridium/ It is made with being embeddable as a primary goal, it is simply a html+js file that can even be opened directly from the file system to edit a notebook, without even needing to host a static server, Now with this embeddable editor ready, the next step was to create a thin wrapper to make it a custom visual for Power BI that adds the linked data as a variable in the dataflow and persist changes etc., https://github.com/wishyoulization/PowerBI-visuals-Iridium
Power BI Demo
Tableau Demo
Now, with a simple trex file (download from: https://wishyoulization.github.io/iridium/tableau-iridium.trex) pointing to the hosted url we can embed as a Tableau Extension, and it’s also possible to create these visuals first on the observable platform with advanced editing and collaboration features and export to use in Tableau, examples here: https://observablehq.com/collection/@a10k/tableau
For a step-by-step guide on using Iridium in Tableau, follow this article.
P.S. I’m an Observable ambassador and strongly recommend the platform for any data analysis or visualization project. If you have not checked the observable platform yet please go check it out right away! And this (Iridium) project is a personal project/experiment to embed similar notebook experience in BI tools and is not associated with the platform.
Thank you for reading! Please share your feedback.
