Embracing a DIY attitude, aka “F*#! it, I’ll make it myself”

Published in

Design, Strategy, Data & People

8 min readMar 24, 2017

I’m not a developer or a data expert, I’m for want of a better term; a designer. I have rudimentary HTML/CSS/JS skills and they are pretty old school (learnt on the job back in the days of table based layouts). I have no solid developer skills or training. However, I’m more than happy to roll up my sleeves and take a stab at hacking something together. This post is about combining that dangerously small amount of knowledge (which is often all you need) with a DIY attitude and diving into building a Qlik Sense visualisation extension.

Working with data, but being a visual person, a while back I had this idea that Qlik Sense would be a great way to analyse my Instagrams and other image based resources. I wanted to browse the images, sort them, filter them and that isn’t something (at time of writing) you can do out of the box with Qlik Sense. What I had in mind was a simple grid to display my Instagrams, essentially a gallery object that displayed the images filtered by any the selections in the data.

After all that’s only a bunch of HTML and some javascript, how hard could it be? I had a quick look on Qlik Branch at what other people had made, but nothing really fitted what I had in mind. So armed with my trusty CMD+C/CMD+V skills (yes, I’m a Mac user), I set out to write my first extension.

The challenges

The first challenge is set up. The easiest way to develop extensions with Qlik Sense is using the free desktop version. But I’m afraid it’s Windows only, so that means you need a VM on a Mac.

My second challenge was understanding how extensions are written. There’s a basic ‘hello world’ extension tutorial , but I’m not a tutorial kind of guy. I’ve always found I learn more if I’m trying to do something real with a new tool rather than some canned throwaway thing (of course, that’s also why I’m not a great developer as I often miss out the fundamentals). What I tend to look at is how other people approach a similar problem, then cut, paste, tweak, observe, change, repeat. It’s not the most efficient practice but it works for me.

Thankfully Qlik Branch is full of interesting extension projects, and best of all, extensions aren’t compiled in anyway so you can simply open one up and taken a look at the components and code. That combined with the Qlik’s developer help site meant I could get started very quickly, and most importantly be working with something that I could instantly see the effects of the changes I made, without the struggle of understanding all the set up code.

After that it was just a case of working out the basic idea for the extension and choosing an approach I felt I could work with. As ever there’s always more than one way to skin a cat and part of a being a good dev is assessing which approach to take — as I said I’m not a good dev, so I went with what I could understand quickest. A simple grid like object, made up of a bunch of fixed size divs that had background images, the file path of which is supplied by a field in the Qlik Sense App’s data model.

The gotchas

Caching. One of things that can catch out non devs like me is caching. With extensions this can be tricky as you have to be aware that both Qlik Sense and the browser are caching things. If you write code like I do (small change, refresh, console.log) then developer tools in Chrome is essential, as you can turn off caching and do hard reloads. To use it you need to access the Qlik Sense desktop through the browser (localhost:4848/hub/my/work) rather than the desktop application itself (but that needs to be open and running in the background). Recently they’ve added a ‘dev hub’ that allows you to work on extensions directly in the client but I like to use Sublime Text to write code in.

Hypercubes. The data in your extension object is not ‘all’ the data in the Qlik Sense app, it’s just what is included in the extension’s hypercube. That is what the user defines by adding dimensions (fields) and measures (aggregations) to the extension via properties (basically it’s the largest number items from the dimensions multiplied by the total number of different dimensions and measures).

The more you allow the user to add, the larger the potential hypercube. However Qlik Sense has a built in maximum of 10,000 before you have to page the hypercube data, and you need to build a way to handle that paging in the extension. Of course it does tell you that on the dev help site (so RTFM to me), as I spent ages trying to figure out why the rest of my data wasn’t showing.

Also just because you can show 10,000 rows doesn’t mean you should. For me each row represented an image that I would load into the extension, so I capped it at 600. Which mean’t 100 images (the height) multiplied by the 6 possible dimensions and measures (the width) to avoid the browser falling over (later I added the paging and the ability to set a custom cap).

External assets. Although Qlik Sense essential acts like a mini web server on your machine, it’s not. You can’t use the rest of your file system as storage for any assets you may need. So all resources need to be inside the ‘Sense/Extensions’ folder or available via a real URL (or at least that’s how it seems). This was/is a headache for me as it meant that all the images to be shown had to be copied to that folder. Not very useful. However you can use anything that you can reference using a URL, so that meant (in my mind at least) the idea was still valid.

The good bits

For the advanced types out there, extension builders can use all the Javascript resources at hand including things like D3, something that has already solved many of the hard issues faced when visualising data. For the less advanced, once you get your head around accessing the data and working with jQuery you can create something useful, very quickly using just HTML elements and CSS.

On top of being relatively easy to get something on screen, it’s already tied into the Qlik Sense App’s data model. It’s instantly integrated, enabling interaction across the App to update the hypercube that is used in your extension. It’s immediately dynamic and ready to use as part of a broader experience or piece of analysis.

The Qlik Sense rendering model, means that the extension redraws when it needs to. So each time the hypercube changes or its container shape changes it redraws to ensure the right data is there and the layout works. With even the most rudimentary understanding of HTML/CSS you can create an extension that adapts to fit the shape it’s been given to display in. Most of which is managed by Qlik Sense’s responsive grid layout.

What’s also really useful is that any options you want your users to be able to set can be easily added to the Qlik Sense properties panel. You can reuse the standard property panes for adding data, sorting etc from Qlik Sense and then add what you need from a range of different controls. Enabling the extension to be tuned to the user’s needs or visual preferences.

The result

So as with all software, my MGO Image Grid suffered from feature creep. The initial version simply displayed a grid of the first 100 images in the selection at a fixed image size of 100px by 100px. As soon as I started using it I quickly I found new things I wanted it to do for me, and thus started to think more about the things other people may need it to do (usually I start with other peoples needs but this was originally just something fun for me).

I’d designed it for square Instagram images, but what if your images are portrait or landscape? What if you want to use a low res version on the image on the grid view (helping load times and memory footprint) and a high res one for when you are viewing just one image. What about using the image as essentially a hyperlink through to a different type of asset, like a web page? How about being able to display measures on the images (such as number of likes or comments). What about the ability to visually flag an image based on some criteria the user can set?

It’s been over a year since I first created and added my MGO Image Grid extension to Qlik Branch and I haven’t stopped tinkering with it (and using it in my Qlik Sense apps). What started as a simple set of properties grew and grew, supporting a wide set of use cases. It’s at version v3 now, has a GitHub page and I’ve just added another flavour that incorporates the Microsoft Cognitive Services APIs. This enables you to do facial recognition, emotion analysis, content analysis and even a little OCR on the images. It’s come a long way from “a simple grid to display my Instagrams”.

Don’t worry I’ll stop before adding instant messaging to it 😉.

What’s next

The developer’s prayer or unicorn tears:
I must refactor the code, so it’s cleaner, understandable and more efficient…
especially as people can now see my code 😱 .

Assets and resources: