The Blue-Collarization of Tech
Frank sits in front of me, on the long side of a long conference table. We’re crammed into a first floor office that got recently converted into a shared space for engineers to plan in. The construction software company we work for is growing so fast that this is the only room available for us to meet in. With only about two feet between each side of the table and the wood-paneled walls of this old building, there’s only enough room for us to move the chairs back and barely squeeze out of them. Brown shoes, blue jeans, and a plaid, collared flannel adorn his stocky frame. The giant window behind his side of the table perfectly frames him in front of the bright green forest shining through as he clicks and clacks away at his keyboard. I can’t help but mistake him sitting in here, compared to someone chopping part of the forest down outside.
He looks up at me from his laptop with a smile poking through his grizzled, beardy face. “We’re still figuring out how to deploy the new app in a way that lets us target the version two API only for our beta build, but I think I’m close to a solution. We’ll need a way to push this stuff through without totally wrecking production when it goes out.”
That’s where I come in. I’m the person who’s tasked with testing everything so we can ship “without totally wrecking production”. Only a few days ago, it was decided that we needed to cut scope on another project to entirely focus on getting this release out in four weeks.
One-hundred and sixty miles northwest in Detroit, I’m picturing an automobile production line screeching to a halt. Dozens of technicians stepping back from the previously moving conveyor belt to take a break while production engineers like me and Frank come in and have the same conversation. “We need to recalibrate this section of the line to account for the new model reaching this point. Can you test this to ensure it doesn’t break when the new chassis gets here? We have four weeks to get it out.” Flannel fits in far better here, doesn’t it? At least, that’s what I used to think.
Frank guides me through the API changes version two contains that I need to account for when I test the new beta. I fill two pages with notes, during our conversation, my own clicking and clacking taking over, while he dictates in the background. I note the tiny tweaks and idiosyncrasies that could cause customers to fail at their tasks. When I’m done typing, I have an entire set of notes that I’ll turn into release requirements that will transform into a test plan that will inevitably become documentation on how this abstract, electronic thing we’re building is functional. Bingo-bango-bongo.
This is the reality of building software nowadays. Coming up with a few new features for the hot, new model, adjusting your production line, sending all these requirements off to a larger team of engineers, and shipping everything off so you can take home a little more bread to the family. Instead of releasing a new model every year, we do it every quarter, every week, or even every day.
Meanwhile, back in the 90's…
It wasn’t always this way. In the 90’s, software was a far smaller, far slower, less-guaranteed living. You had tremendously successful, big companies like Microsoft and Apple, but the industry was still primarily built around longer planned releases from smaller companies that charged customers for each one. Software jobs were still seen as extremely white collar, even when servicing smaller, blue collar industries. A few nerds would be the people enabling new methods of tracking shipping analytics for manufacturing companies, while the boots-on-the-ground workers themselves were seen as the normies that made physical products for the other normies. Software was, for lack of a better term: special.
Nowadays, manufacturing companies and software companies are largely the same. Through the spread of modern web services, “agile” practices, and mainstream startup culture, the industry has transformed into the modern day equivalent of factory work. Instead of in some giant building, your factory floor is in the cloud. A laptop giving you access from your cushy couch or office chair at home. The conveyor belt is a continuous integration and continuous delivery (CI/CD) application like CircleCI or Jenkins. The foreman, a product manager, grooming little deliverables that other line workers will guide through a production line from sprint to sprint.
You could certainly make the case for this back in the 1990’s, when big box software was still the big, stomping beast in the yard, but the yard was still largely one of a white-collar, financially demanding industry for people that still retained some highfalutin sense of business. The cultural and technological framework still in its infancy in relation to this impending factory type of paradigm. Since solutions were slower to develop and ship, the cost of business stayed quite high.
The Creation of the Online Factory
You may be asking, incredulously, “Uh-huh. Sure. And how exactly did this move to blue collar work happen?” Glad you asked! Let’s start with the technology.
The Gift of the Internet
Internet-connected home computers were already becoming ubiquitous throughout the 90’s. The spread and subsequent cost reduction made this the hot, new thing for the suburban, American household. Adoption of internet devices would only snowball after the debut of iPhone in 2007 put a smart device in everyone’s pocket. Meanwhile, cloud technology was on the rise with on-premises web services being replaced by cloud solutions that let companies subsidize their own server costs with others using the same infrastructure.
Web pages and apps, previously local-running dummies that would do basic calculations, were moving towards increasingly using public APIs running in the cloud. As unwieldy SOAP APIs gave way to more public, self-documenting REST APIs, the internet changed rapidly as new data wells proliferated throughout the world wide web as we know it. Static web pages were quickly replaced with more dynamic datasets that used these APIs for performing more intricate operations. A basic page of truck stats could be replaced with a shiny new version that gave you live analytics on your computer. Or — even better — right to your iPhone with the push of a button.
The best part? Most of these APIs are written the same way every single time. You get some data, you post some data, you delete some data, or you put some data. Some companies, certainly, do far more elaborate operations, behind the scenes, but most of them rely on exposing very simple data to their end users that get delivered from release to release.
Previously, an industry built on big box software development that relied on yearly release schedules was replaced with an early iteration of this online factory line deploying these APIs and minute changes to them quarterly, weekly, or even daily. This created new schedules that aimed at constant-running cloud software delivering tiny, new bits of information from release to release to better serve these dynamic web apps or iPhone apps end-users were increasingly using.
A More Agile Way of Doing Things
It’s important to note the new process changes I glossed over that further instilled this new factory-line mentality as apps and APIs became more wide-spread. With release schedules getting faster and faster, a new form of engineering dubbed “extreme programming” rose throughout the 90’s and early-2000’s. Older engineers may recognize this as the form of programming that would directly lead to agile practices and what we now know as modern scrum. For younger or less familiar engineers, this is the genesis of your standup meeting. This is your sprint schedule. This is your kanban board.
In fact, let’s see the definition of kanban, according to Dictionary.com real quick. No real reason. Let’s see. “a just-in-time method of inventory control, originally developed in Japanese automobile factories.” Huh. That might come back in a minute. Anyway, back to what I was saying.
Just as the groundwork was laid with web technology moving into a model more resembling the delivery mechanism of factory work, the process to create that technology was also adapting to that model with agile. Or scrum. Or whatever you want to call it now; it doesn’t matter. The point is that this new framework built around these tiny deliverables slowly drove teams to develop software just like it was on a production line. Your daily standup meeting a checkpoint for the next set of gates you and your team would tackle to get your deliverable through the line and out the door. Your developers building a window or a door and sending it to quality assurance for testing. Giving your product manager, the foreman, a last look at the item before it crosses over into the real world the following day.
Some Quick Perspective
Before we move on, it’s important to understand similar changes that happened in manufacturing throughout the 20th century, for the sake of comparison. Early factories were extremely rudimentary institutions. Workers lining facility floors and working through each stage of the manufacturing process manually as systems moved product in little chunks. Much of this work was dangerous and accident prone as manufacturing grew faster and faster without proper regulation to keep up.
Increasing automation and process changes led to our current manufacturing process and what we consider a largely blue-collar industry of Americans building everything from automobiles to blenders for everyone. Similar to the automotive origins of kanban, lean manufacturing practices were created in the 50’s which sought to improve efficiency by integrating humans into the system-level mindset of the process. Funny enough, there’s a related practice called just-in-time manufacturing, a term later adopted by Microsoft for their usage in their just-in-time (or JIT) compiler. Similar to kanban, the basis of which comes from the automotive industry. Neat.
Standards + Regulations = ?
The only major difference, as I see it, between software and a more blue-collar endeavor like manufacturing, is that of regulation and standardization. American industrialization swept through like a wildfire, planting factories across the nation at a blistering pace throughout the twentieth century. Our necessity for products supplanted our common sense at how safe these institutions were as they spawned. This would cause a number of safety measures to be enacted that would shape our modern view of the industry.
Zoning laws, for example, were created in response to construction in towns throughout the US that had issues with noise, health and even crime surrounding these places. OSHA and the EPA were created to push for safer business practices when and ensuring basic worker safety was enshrined in common legislation. Sadly, some front-line workers would be guinea pigs for greater society, discovering health issues such as cancer and physical issues throughout their tenure. This would lead to a great crackdown, resulting in safer, but more measured versions of these companies operating.
Should software engineers worry about dropping dead at their desk when they check new code in to production? Of course not. But, you’d be naive to assume election rigging, social activism, security breaches, and surveillance don’t have real world impact directly influenced by software.
Unionization efforts can be looked at in much the same way when compared to manufacturing. We’re seeing roles positioned uniquely at the lowest rung in the corporate totem pole (It’s ok, I can say that. I’m a QA person.) work hard to build some better guardrails around their rights in the software space. Software was significantly more democratized in it’s hey-day. Did people still get screwed? Of course. The difference is that so much money was flowing through the industry in every layer of the business that much of the frustration was avoided. Only now do people like support staff and QA get undercut in favor of executive pay boosts annually for most companies.
I imagine many engineers are worried about a lot of these new, more blue collar aspects of our industry due to the financial security currently provided by software. They’ve been paid a lot, given a tremendous amount of creative freedom, and had very little to worry about in terms of regulation in our roles up until now. Will this ruin the industry? Will you get paid less? Will your boss be more worried about what you’re doing day to day as these potential standards drive more accountability into our roles? To me: who cares. Plenty of blue collar workers make a decent living and can provide for their family. I think the concern from a lot of engineers when I bring this topic up is more of perceived affluence vs. the reality we live in. Software isn’t special anymore. The technology is now a common endeavor, the people that build technology are also becoming more ubiquitous. There’s nothing wrong with that.
Back to Frank and I
After an afternoon of testing, I come in the next morning to an email with a meeting invite for 10:00 am with Frank and the engineering manager, Louis. For the uninitiated, Louis is the prime example of a pretentious engineer. Condescending tone. Comes in late, leaves early. Most meetings are monologues vs. dialogues on how to make good software. He’s extremely adept at organizing people and finding quick solutions to problems, however, so his post isn’t entirely unwarranted.
As it turns out, we’re cutting scope yet again. Our four weeks to release is now turning into two weeks. The hope is that we get the product out as quickly as possible for a customer that’s begging to use it. They’re willing to start paying for it now in order to offset the cost of the work we’ve done so far. I requested two important security bugs get patched before we ship, but those are being dropped in priority. I’m told we’ll fix them later, after the customer starts using the product. The goal now is to find any larger last minute bugs and regression test, in order to ensure the customer can use the software.
I make my issues clear to Frank and Louis, but I’m dismissed and told the software works just fine. This isn’t even getting into my personal view of what we’re working on. It’s trash right now. The UI is still buggy and needs another pass or two. We haven’t even done a usability pass yet to figure out if normal humans can even engage with the ideas we’re presenting. The backend has performance issues when users configure large lists of data. It “works”, but it’s a mess.
I schedule another meeting to bring my concerns up to Louis again in a private meeting and he immediately scoffs. Without Frank there to back me up, I get even less of an opportunity to make my case that we need another two weeks (at least). Next, I go to my manager with the same concerns and am told Louis is the expert. I even set a meeting with the CEO of the company (yeah, I’m one of those guys) and I’m deflected yet again and told I can find new work if I can’t fall in line and help get the product out the door. The meeting ends with one of my favorite quotes in my career. As I’m walking out, the CEO puts his arm around my shoulder and tells me, “Alright, thanks for the concern, but get back to work. Just let it go. Nobody’s paying you to care that much.”
It’s not all bad, everyone
There are growing pains in this transition. As illustrated above, not everyone sees themselves in the customers they serve, delivering these amazing products to end-users hungry for safer, more efficient modes of working. Louis is the prime example of a continued pomposity I see quite often in the industry. I think it has to do with the way a lot of “classic” software engineers were educated. They were the master of their domain. One of a few, special software developers that could make technology that most people couldn’t even fathom. Nerds providing the truck analytics to the normies.
But, it’s not that anymore. And that’s a actually a wonderful thing. Technology is being used and developed by more people than ever now. In the 90’s, a nerd like me was looked at as a social pariah for understanding how the computer thing worked. As an industry, we’ve gone from around 600,000 in 1990 to almost 5,000,000 strong as of this year (2020). People like Frank entering the industry through coding boot camps spread throughout the US that enable people living even in more rural parts of the country to manufacture software solutions like others in the automotive industry have been for years.
We only need to be careful to better regulate the solutions we’re building. Ensuring the customers we serve can do their jobs safely and efficiently, without worrying about data breaches or having their health data tracked and shipped off to some company somewhere.
Continuing to lower the barrier to entry for the industry, regulating what we build, and leaning into the simple, information as deliverables aspect of our roles will only make software better. We got a little lost there in the 2000’s, because of the quick adoption of technology without ample regulation, but we have a real opportunity to course correct now. We should usher in a new era of tech that’s both accessible and continuing on the trajectory towards a more standardized, smart way of building software by everyone, for everyone.
—
Parts of this essay are based on my real experiences in the software industry. I’ve changed all of the names to better focus on the overall story and not hang any of these people out to dry. Even the more annoying of those I’ve written about are good people, deep down, and deserve their anonymity.
*Note on stats chart. I sourced numbers from the references below. Since the data was all presented a little different, I took some janky liberties. “Occupations in Information Technology” presents the total IT workforce with a breakout for percentages of IT jobs that are software in nature. I halved the IT jobs for the years presented in the article as a shortcut to ‘tech’ or ‘software’ jobs I’m looking at. There’s probably a better, more accurate way to find that data.
References
- “Occupational Changes During the 20th Century”, Bureau of Labor Statistics, Ian D. Wyatt and Daniel E. Hacker
- “May 2021 National Occupational Employment and Wage Estimates United States”, Bureau of Labor Statistics
- “Occupations in Information Technology”, United States Census Bureau, Julia Beckhusen
Recommended Reading
- “How factories were made safe”, The Roots of Progress, Jason Crawford
- “Where Did All the Workers Go?”, The Atlantic, Derek Thompson
