You Should Reconsider Training on Prod
“The patient and the textbook should not be opened up at the same time”
— Kent Beck
I’m beginning to think that I missed the meeting where it was decided that every single job posting had to include “ship code on your first day!” As job posting spam goes, it’s pretty alluring — a sort of shibboleth for “fast-moving” companies. We’re not just Agile, we’re agile.
It’s so convenient as a shorthand, in fact, that many companies don’t even evaluate whether it’s a good idea. There are upsides and downsides to onboarding in production, and we have good reasons to think them through.
The Good Things About Training in Prod
There are some solid reasons to throw new hires into the pool, and it’s understandable why this practice is so pervasive:
Training in production forces you to keep processes lean, and it acts as a sort of canary in the coal mine. If your platform is made of duct tape and hope, new employees won’t be able to ship on day one — pain which will hopefully force you to fix problems. (In reality, many companies stick their fingers in their ears, or they carve out exceptions for small day-1 bugs.)
The practice also feels consistent with what new employees want. Changing jobs is already stressful, and new hires want to be useful as quickly as possible. Shipping early is an obvious way to alleviate these fears.
It gives us a jump on teaching newbies about our business domain, which will have to happen eventually anyway. We hope that by reading the existing code, employees will start to get an early understanding of what we do.
And if nothing else, it’s a strong signal to potential hires. Developers working for companies with heavy processes see these ads and get teary-eyed and wistful for the days when they could ship without a Letter of Marque from the Architectural Committee. It seems like an obvious win.
The Bad Things About Training in Prod
The drawbacks of this approach, though, are serious. Enough so that for many companies it would be better to delay production work.
Given the general breakneck pace of our industry, a new employee’s first days are probably the last ones where they won’t be overloaded with looming deadlines. And it’s in that time that they have the opportunity to fill in gaps in their knowledge — later on their learning is likely to fall by the wayside or be pushed into whatever empty evenings they have.
If they avoid that pitfall, an even more sinister one awaits them: most production codebases reinforce questionable programming practices. We should expose new employees to good examples, not bad ones. Especially not bad ones that lack the context necessary to understand them. By exposing people to our worst decisions, we create bad habits from the start.
This is compounded in companies where there is a commitment to push code into production on that first day. This creates a pressure to ship code regardless of its quality. You probably won’t turn down my marginal code, because as a nice person you want to “give me a win”. Marginal code that was developed, I might add, without even a day’s worth of business context.
It’s a major missed opportunity. New employees have the potential of raising the bar for everyone in an organization and bringing in new perspectives without being weighed down by attachment to old decisions. To do so, we need to push them beyond the trade-offs and mistakes we made as a younger company. We can’t do that while mired in those mistakes.
And finally, importantly, immediately pushing employees to ship production code is intimidating. If you think that’s not a problem, you are mistaken. I’ve spoken to many employees about the training programs I’ve created, and some fantastic hires have told me that their Impostor Syndrome would have precluded them from even applying were it not for training.
This is a huge diversity issue. The strong signal we wanted to send may be excluding some of the best people we wanted to attract.
Can We Split the Difference?
Yes. Maybe. All things being equal, it is much better to ship sooner than later. Just not at the expense of improvement and solid foundations. I do have some ideas on how we can get the better parts of both worlds.
First, remember that while shipping on day one is an indicator of agile, build automation, and so forth, we can have those things without that one indicator. There are other ways for us to keep ourselves honest.
Looking at the upsides and downsides, small corrections can solve a lot of these problems. Have employees do non-production training, but model your training exercises after your actual business. When trainees start in on the production system, pair them with someone for a few days who can explain the business.
It’s often hard to justify training on the entire tech stack before switching to production work (though I recommend it in some cases). But we can at least identify particular gaps that our non-training onboarding leaves, and address those skills specifically. If your team does a poor job at idiomatic testing, at least train specifically on that topic.
Finally, if we build a strong learning culture for our companies, we can continue learning on an ongoing basis, raising the bar with existing employees as much as with new ones. Education programs reinforce to our employees that we’re willing to invest in them long-term. That’s good for engagement and retention, and that makes it good business.
