Tech Nation 2017: At the forefront of bad data

Earlier this year Tech City released “the most comprehensive analysis of the UK Digital Tech Ecosystem” and, unsurprisingly, I had a little rant on twitter. Twitter is great for short little bursts of anger or frustration but it’s not the best medium to adequately explain my frustrations with the report, so here we are.

Should Designers Code?

I’m going to resist the urge to criticise all 124 pages of the report but I want to elaborate on some of the key issues that I believe invalidate the entire analysis.

My primary criticism of this report is the claim that the number of Github users is an appropriate metric for measuring the landscape of digital tech skills. The tech landscape in London comprises of a vast range of skills that extend far beyond software development. Software development teams almost never exist as a sole entity. They are typically surrounded by a team of Project Managers, Product Owners, Digital Designers and so on. It’s the combination of those groups that result in companies producing world class products and attracting ridiculous Venture Capital investment. Some of those people have Github accounts but the vast majority don’t. Let’s play devils advocate for a minute. Let’s assume (take a deep breath) that the people behind this report believe that developers are the only people that matter (now breathe). Github users aren’t obligated to set an accurate location on their profile (my own Github account is a perfect example: https://github.com/TalentStuff). Even if you assume that only a marginal percentage don’t have their location set in their profile, what percentage have their locations set as ‘United Kingdom’ or something similar despite the fact that they live and work in London? What about the percentage of developers in the UK that don’t actually have a Github account?

Using Github location as a metric to measure the size of a tech landscape clearly fails, even if you’re only focusing on the total number of developers, but as I alluded to earlier, by only measuring the number of developers you are failing to come even remotely close to the true number of ‘digital tech skills’ in any given region. If the report was even remotely accurate, then surely the total number of designers would have been factored into the equation? I suspect the reason the people behind this report focused exclusively on developers is because compiling accurate numbers for a landscape as diverse as tech is incredibly difficult. Taking blatant shortcuts like this calls into question the validity of the entire report.

Inclusivity & Privilege

The concept that a university education is a suitable metric of Talent is a controversial and highly debatable point worthy of its own post and I’m not going to go down that rabbit hole just yet. That being said, it’s still worth addressing the logic behind its inclusion in the Tech City report.

The Tech City team know better than most that the tech industry struggles with diversity and inclusion. This industry is more accessible to middle-class white men than it is to more underprivileged groups. Tech City have been vocal about this fact for quite a while (as they should be) and have led and supported a number of great initiatives to help address the issue. Much to my obvious chagrin, Tech City are an incredibly influential organisation. Companies look to Tech City for guidance and direction and by promoting the concept of leading Universities, they are perpetrating antiquated and exclusionary biases that will lead some employers to double-down on the concept that ‘better’ candidates come from ‘better’ universities.

As they stated directly in this very report:

Digital tech companies could do more to attract and retain women as well as to challenge stereotypes and cultural biases.

Promoting the idea that better candidates come from better universities is an exclusionary stereotype perpetrated by cultural biases. Tech City are more obligated than most to help promote diversity and inclusivity in this industry and that’s not just limited to improving gender diversity and, to be honest, I expect them to be better than this.

Meetup is Terrible

I’ve been running one of Europe’s largest tech meetup groups for about five years now and we have been running the event entirely through meetup.com. I can personally attest to the fact that any data garnered from meetup.com is just terrible.

The primary issue with the meetup data used in this report is not too dissimilar to the issues with the Github data. How do you define a ‘tech meetup’? The meetup that I help organise wasn’t recognised as a ‘notable tech meetup’ despite being the largest of its kind in London.

Let’s cut them some slack. Let’s again assume that accumulating accurate data is just too hard. Gathering data on their own network is surely much easier right? Afraid not. The report claims that Founders Network ran 8 workshops for 400 tech startups in 2016/17. Sounds great right?

The truth is, Founders Network ran 18 workshops and 1 conference over 6 months, reaching 500+ founders across the North.

Data is Hard & Only VC Backed Companies Matter

It should be obvious by now that the biggest issue with this report is the wildly inaccurate and cherry-picked data used. I’ve not even mentioned the two most blatant inaccuracies.

It’s not too hard to define what industry a new company belongs to thanks to Companies House. It is hard however to define the difference between a new tech company and a developer or a designer establishing a limited company in order to work as a freelancer. My guess is that if you eliminated all the ‘companies’ established by people wanting to provide a freelance service, the ridiculous claim of ‘1 tech company is started in London every hour’ becomes dramatically less impressive as a soundbite.

In 2016 the UK secured £6.8 billion venture capital and private equity digital tech investment — that’s over 50% more than any other European country.

I’ve no idea how accurate the numbers in the quote above are. To be honest, I don’t really care either. The reason I’m highlighting it is because it’s one of countless statements that hammers home Tech Cities point:
If your company doesn’t owe millions of pounds to Venture Capitalists, then your company doesn’t count.
As Stef Lewandowski pointed out on Twitter, the original survey this report was founded on was predicated on the idea that a tech business should be following the VC-funded model. The questions were irrelevant to the people running bootstrapped companies, therefore we can only assume that they are yet another significant and important group that weren’t included in the ‘digital landscape’ of this country.

So… Now What?

I don’t have the answers to the questions I’m posing above. I can only hope that someone from Tech City will read my concerns and take steps to address them before they publish the Tech Nation Report 2018. If you work for Tech City and want to talk to me about anything I address in this post, I’m happy to do so both privately or publicly.

To give credit where it’s due, Eileen Burbidge (a person I have an enormous amount of respect for despite this report) did respond to some of my concerns on Twitter and I’m sure she agrees that gathering accurate data is incredibly difficult.

The fact is, Tech City is a Government organisation who “strive for excellence in everything we do”. As they are the most prominent voice for the UK tech industry, I really don’t think it’s unreasonable to hold them to a higher standard and present a report that isn’t fundamentally flawed and often just plain wrong.