The Case Against Self-Service Data and Analytics

Kat Hempstalk
The Startup
Published in
7 min readFeb 12, 2021

Damn. You just discovered the latest sales report doesn’t include the new category of product and you know it’s going to take forever for the always-busy internal analytics to make the tiny wee change to add it. Why can’t they just let you just do it yourself? In fact, you know what you’re doing and have authority, so you have them to add you to the tooling and you do exactly that. Done. See, now that wasn’t so hard, was it?

Sound familiar?

It should. I am yet to meet an analyst — or someone who works with one — who hasn’t felt this pain at some point. The answer typically is to give the people who ask for access, access. Once upon a time, I would have agreed with this and I would have been the person asking for access too! But, I’ve since come to realise that going all-in on self-service is the wrong approach.

Why data is walled off

Let’s take a step back and discuss why the data and analytics was walled off in the first place. Reasons might include:

  • Security: privacy, security, and contractual requirements might mean access to data is limited. Best-practice security usually requires minimum access: only those who absolutely need it to do their jobs get access.
  • Structure: the data and reporting simply isn’t organised well enough to make it easy or safe to share.
  • Tooling: there aren’t the tools available to work with the data or reports. Or, even if the tooling is available it might be excessively expensive or challenging to work with which in turn limits the number of people who you give access to.
  • Literacy: the data is organised well but there’s a lack of trust that you will treat it with due care, giving more people access would risk the ‘cleanliness’ or integrity of that structure. Also, there may be a lack of trust, or a genuine lack of competence, that you know what the data means or how to correctly analyse it. Giving unfettered access will therefore endanger decision making that may be based on the data or reports.
  • Interest: no one cares about data enough to want to look at the data themselves, they would much prefer someone else did it for them.

These reasons are all good reasons to limit access to data and analytics: they ensure your organisation is able to make sound data-driven decisions, without putting the data (or organisation) at risk.

A real fortress. Well, an old one that was easier to find than a data fortress.

The impact of a fortress

However, even though there might be great reasons to limit access to data and analytics, new problems are caused by doing so. The most common problem I’ve experienced is an analytics team who simply can’t keep up with demand: they have so many requests for reports they have lead times of weeks for tasks that may only take minutes to complete. In turn, this generates a behaviour amongst analysts of wanting to be the knight in shining armour that rides in and saves the day by turning around a task quickly, or by allowing their friends to jump the queue. It may also have the opposite effect: analysts get so tired of repeating the same tasks over and over they get bored and quit.

You might find you constantly need to keep adding analysts to the team, but even as the team grows the backlog never seems to shrink.

With a fortress built around data and reports it may also be difficult for those needing it for decision making to get information in time — so even though the data is eventually provided and is correct, it’s not used because it is provided too late. Decision makers may lose interest in working with the analytics team because they are viewed as ‘slow’ (even though they are always busy!). Instead, they begin to make decisions on ‘gut feel’, and may lose interest in the data entirely.

Lean / agile may help…

Kanban board
Kanban board. Image courtesy of Wikimedia Commons.

… but prioritisation is not the only problem. We managed to speed up our central analytics team at one organisation by improving our planning process. We limited work in progress and used cost-of-delay to prioritise tasks, and worked off a Kanban board with daily standups, and weekly grooming and retrospectives. It had a great effect on analyst behaviours: because there was now a defined process for prioritization it was difficult for jobs to jump the queue, or analysts to pick easier/smaller tasks to look like heroes. It also caused us to improve some of our lead times — particularly for smaller and more urgent tasks — but ultimately the size of the backlog was still ridiculous and the problem of data being relatively inaccessible still remained.

Self-service to the rescue?

The answer at this point seems easy: to lessen the load on the analytics team, just open up access so people can sort themselves out. This means less tickets coming into the team so they can work on the ‘hard’ stuff, and the smaller jobs go away and take care of themselves. And they do.

But then again, they don’t. Short term you may have solved the speed which which you can make decisions from data, but long term you might find:

  • It’s difficult to audit who has access to what (and whether they should), and whether they have accessed data they have permission to. For example, giving access to the server containing all of the data warehouses might mean you have a simple permissions model, but perhaps there’s a table with personal data in there and you’ve unwittingly given an employee a means to stalk an ex. (Ok, this is kind of worse case, but stranger things have happened!)
  • Your ‘clean’ data store suddenly has hundreds more tables/files/etc, no one knows what half of them are for but you’re too scared to delete them in case something breaks.
  • You’ve blown your budget with tooling: suddenly you’ve gone from 10 licenses to 1000 because everyone needs access.
  • Analytics ‘silos’ form: the manager in another part of the org who now has access realises they are wasting too much time trying to do it themselves, so they hire someone person to do it for them. That person at best might keep that manager happy, but at worst is a law unto themselves who leaves behind a trail of destruction (including incorrect reports, poorly written code, and their own data silos) and ultimately more work for the main analytics team.

And, most likely: even if the underlying data/analytics is standardised, you end up with several versions of ‘the truth’. This makes analysts sad. One of the reasons data-driven decisions are powerful is because they are objective. You are able to make a decision based on fact, bringing in emotion by choice. When several versions of ‘the truth’ exist decision making becomes a more emotional decision about whose version to trust. And as most analysts will tell you: you can torture data to say whatever you want. So, the data-driven decision making in this scenario is actually the loudest-(or most highly paid)-person-in-the-room decision making.

If not self-service then what?

It’s a balance. Self-service is great, but it needs guard rails: protections in place to ensure it achieves the outcomes intended without creating more problems along the way. There is value in structure and security: ensuring not just that the data is safe, but the decisions made from it are sound, too.

I have been fortunate to work in a number of great technology companies and each has been at different stages of data and analytics competency — as I too have! But, the answer for me has always been about relationship building.

Two people talking
Image courtesy of the Noun Project.

As an analyst there was value in collaborating: in retrospect if I had spent more time working together and sharing with other analysts we would have found the commonalities so work wasn’t repeated. We could have easily standardised and automated common reports if only we’d bothered to talk to each other rather than independently working on tasks. This would have removed the repetition from our backlog, and generated a better culture amongst the team.

As an data and analytics manager it’s worth taking the time to build the relationships with not just peers — but more senior and junior people too — and understand what outcomes they hold most dear. Understanding their needs makes it easy to draw a line (with them!) between what goes data needs to stay in the fortress, and what can be released for general consumption. Once you have an agreed set of data / measures / graphs / reports / dashboards with agreed access levels and tooling, it frees up the data team the data team to work on more interesting tasks and create more value for the organisation. And when you have a relationship, you’ll both be more keen to help each other get to where you want to go.

Building relationships can be hard for analysts as many of us are introverts and other people are scary. But, start small with one person and see how you go. The alternative is probably more daunting!

--

--

Kat Hempstalk
The Startup

Machine Learning, AI and Data Expert. By day I work training machines to think, by night I plot to take over the world. All views expressed here are my own.