The rise and fall of the Dungeon Master

There is a common anti-pattern in software development that is hidden in plain sight, just waiting for someone to name it.

The time has come and the name is “The Dungeon Master”.

Whenever you embark in an enterprise software development project that involves rewriting some portion of a big legacy system, you might bump into a Dungeon Master. Here are some of the distinctive traits.

• The Dungeon Master is the author of the original software, that used to support the business in the early days, and still does (albeit with some headaches).
• He’s deeply entrenched with the software, in facts he knows the software better than anyone else. Even if he’s not any more a software developer, he’s still at close distance from its creation.
• Whenever there’s a problem in that software (and that happens) he’s the one to talk to.
• They usually are surrounded by minions, the complementary anti-pattern that deserves a post of its own (coming soon).

The dark secret of the Dungeon Master is that he knows every trap in the existing legacy software, because he was the one to leave the traps around.

This isn’t intentional or evil. Software is rarely finished-finished, and the moment you have a better insight, you don’t necessarily have the possibility to refactor in order to match the new learning. Knowledge, in the form of accidental complexity starts accumulating in the head of the Dungeon Master, and silently grows.

Invisibly but steadily.

The net result is that for every major change, the Dungeon Master must be involved. He’s the only one able to guide developers across the minefield. Or so it seems.

Having a Dungeon Master as Product Owner or Domain Expert is a terrible idea

Cyrille Martraire exposed it very well during this year’s DDDEurope:

https://twitter.com/ziobrando/status/692993841515909120

Let’s now be brutally honest: after many projects where the former developer of the existing solution played a role in the replacement development project I have to expose the bare truth:

I’ve never seen this approach working.

The main reason is that Dungeon Masters are put in a totally contradictory situation:

• On one hand, the replacement is needed because the company grew, it has now more budget and this is because the legacy software made it possible. In other words the DM did a great job carrying the business on his shoulders. Self esteem is pleased.
• On the other hand, the replacement is needed because we are now hitting the limits of the original approach, be it the model, the technology, the market evolution or the understanding of them all. Releasing a new solution, way better than the original one, will prove that the DM did a bad job. This doesn’t have to be true to be a problem. It’s the unconscious fear of this to happen (Imposter syndrome, maybe) that drives these unspoken thoughts. Imagine the awkward feeling of walking through armies of employees commenting “Oh, finally some decent software!”.

It’s not a generational issue. I’m 45, I’ve encountered DMs that were way younger than me. It’s more an attitude problem, but mostly a system problem. Good developers turn into Dungeon Masters because this is what the system demands. No ethic. No judgement.

Asking an individual to act against his/her own interest is a subtle form of torture. And this is what is going to happen, under the hood.

Now the question is: do you want to be led by someone whose brain is so torn apart?

Because this is exactly what will happen:

• at the conscious level Dungeon Master will join meetings and provide expertise to new developers; he will eventually drop too much knowledge mixing up domain knowledge and his own interpretation, often in the legitimate pursuit to save some time.
• at the unconscious level, new ideas from the development team will inevitably put him on a defensive stance. After all I’ve been thinking to this problem for years, who are you to propose a ‘better’ solution in a few days? or, put in another way, smarter ideas will make him feel or look stupid.

If you haven’t, please give a look to The secrets of consulting by Gerald M. Weinberg, a lot of these dynamics are exposed in the first pages.

Let me clarify better what could happen in the possible outcomes.

1. The new project is much better than its predecessors, allowing for better revenues, and many interesting side effects. However this will also expose how bad the legacy software was and how much money was lost in keeping it alive longer than needed. In a healthy company culture, this won’t be necessarily a problem. It’s part of learning the business. In some cultures this is not acceptable.
2. The new project is slightly better than the legacy. The ego is somewhat safe, but then the whole thing will start to look awkward: was it really necessary to rewrite the software, for so little advantage? However, a little improvement, especially if coming from new technologies that weren’t available before, is probably the sweetest spot.
3. The new project is a failure, an endless bloodbath leading nowhere. This is the best option for the Dungeon Master ego, maybe the company wallet will be hit, but a failure in improving his creation will see him triumphant. Despite a bigger budget and armies of developers nobody could be better than he did.

This situation puts Dungeon Masters in a Dr. Jekyll and Mr. Hyde situation. Despite the best intentions, their help will be little. Or maybe have negative effect.

If the problem wasn’t technology, but the model or the understanding of the problem, then the help from the Dungeon Master can be a poison pill. They’ll usually force the new development team to rerun his modelling process: spoiling all the fun, and eventually repeating his own mistakes, and overall making the project not the best place to be.

Looking at the learning process supporting the software development, we can spot a significant difference: the original learning was mostly discovery, while the second round will tend to be conformity to an existing model. With the best intentions (not reinventing the wheel, saving money, etc.) developers of the new system will be deprived of the possibility of healthy learning.

Now, readers are probably looking for solutions. And I don’t have them. This post was mostly driven on the frustration of seeing recurring dysfunctions happen over and over, and then end up blaming people or unfortunate circumstances.

Here are a few common strategies.

Pretend is a technology problem

Companies can smell the problem, but maybe can’t diagnose it right. Blaming a technology now becoming obsolete is a cheap scapegoat.

• Good: new technologies were not available at the time the software was written. Using new stuff, like microservices, could allow approaches that were not available at the time the software was written. Whether this is actually true or not, the honour is safe. (Clarification: I am not recommending microservices or any other technology approach, just using one, as an example of a technology fresh enough not to be available. Please, don’t use this one as s Straw Man)
• Bad: in practice, this often leads to an expensive rewrite of the existing codebase, with little or no improvement of the actual system behaviour. Most of the time, the generated value is negative: new technologies enslaved to legacy constraints create a sense of frustration in the development team that often leads to more severe troubles. After seeing tons of money flushed developing large SOAs that ended up wrapping legacy databases, I don’t believe this works.

Think Money First

Often the DM has already been taking some management responsibilities, or even some stakes in the company. It is a good thing if the DM starts thinking business first and worry only about Return on Investment and Cost of Ownership.

• Good: hypothetically, from his management role, the Dungeon Master is now detached from mundane things like table structures, primary keys and validation criteria.
• Bad: he is not. The attachment to the solution is still there. Developers installing a Message Bus to replace a database trigger will be a cruel torture. But money tells a different story too. The moment a different implementation shows that a lot money could have been saved, then the cost of keeping the old software for too long is exposed too. And this can be scary: imagine you discover a wrong design decision you made 10 years ago costed millions to your company… would you happily expose it?

Make them part of the new team

A dreamy way out could be to offer them the option to enter history, being part of the winning team that delivered the old version of the software and of the new team too. Like athletes with a career that spans generations.

• Good: in theory, this could revive the best instincts. These people are smart, and being part of a new game could make them unleash their best energies.
• Bad: the amount of re-learning is going to be huge. They are going to find themselves less productive than their younger peers, good in useless things, but embarrassingly slow in what matters today. The temptation to fall back to “the old ways” or “the dark side” will always be there. If DMs climbed up hierarchy in the meanwhile, this is not going to be viable.

Bypass them

Acknowledge from the beginning that their contribution is going to be small or negligible. Leave them the option to play the role of “the grumpy old man” commenting the outcome of new generations of developers, but basically bypass them by opening a direct communication channel between today’s developers and today’s stakeholders (which is a good idea anyway).

• Good: you may discover what the real problem is, without the need to be conformist to yesterday’s solution. More insights can emerge, from conversation happening now eventually leading to a different product, more aligned with the current needs.
• Bad: the old software would still be around. Collaboration will need to happen even in order to bypass systems. Treating an internal piece of software like an uncontrolled third party one could put project lead under scrutiny, while the opposite could open the door to sabotage.

Separate paths

I am not suggesting to hire a hitman, but to acknowledge that the project would be better off without the Dungeon Master around. If you’re looking for a paradigm shift, you’ll need a team with a different attitude and emotional bonding with the legacy.

• Good: the team can now look for the best solution available, no matter what, and eventually implement breakthroughs from alternative approaches. The project is now on a learning path, instead of a conformity path, and the mood will reflect it. The Dungeon Master is now free from the curse of maintaining the legacy and can finally go back to life.
• Bad: a big theoretical money saver (we already have domain expertise) is gone. The need for re-learning is visible, and will affect project estimates. Whenever a bad decision is taken (and this will happen, since the team will be learning), the ghost of the Dungeon Master will appear in the room, warning that things should have been done differently and playing endless “what if …” mind games with the key stakeholders. Not having DMs as part of the team won’t guarantee success (well, …nothing does). It mostly depends on the size of the dungeon, but on many other factors too. In general, this won’t be an easy decision. There’s gratitude and there’s business. And they don’t always go together.

Turn DM into “pure stakeholders”

A lot of the problem revolves around the Product Owner role. Too much power in these hands. Having a Dungeon Master fit into a stakeholder role among many would probably mitigate the problem.

• Good: stakeholders need to pass through the PO in order to steer development, so they shouldn’t micromanage the backlog, and also be forced not to mix explicit and implicit requirements. Only what is “official” will be delivered.
• Bad: whether this approach is workable depends a lot on the people involved, and on the organisation (internal project? contractors? what is the current role of the DM?). It could turn again into a power struggle or allow subtle forms of sabotage.

What if YOU are the Dungeon Master?

If you recognised yourself in this portrait (guess what, I did recognise myself in a couple of traits too), you’re probably going to be disappointed, because I haven’t given many workable solutions.

But actually, your fate is in your hands, and here are a few tips that can make your life easier.

Let it go

You did a great job. You did your best given the constraints and the things you knew at the time you wrote software. You probably did also a lot of things by yourself, because Pair Programming wasn’t in the cards then. So thank you! However, while you’ve been busy doing stuff, somebody else had the privilege of experimenting with NodeJS, microservices and a lot of cool stuff you probably won’t be able to cope with, because you don’t have all that free time anymore. (Clarification, again. This one made a lot of people portray me as a young developer advocating technology as a silver bullet. I am actually the opposite: a former developer acknowledging that I won’t have anymore the privilege of being as good in a given technology as I used to be when I was writing code most of the time).

The new team will do crap. They’ll be wasting time, they’ll implement wrong solutions along the way, or less sophisticated. That’s fine. They’re learning. And that’s a step that can’t be skipped or accelerated, only facilitated.

Start something new

The real fun is in the early days, with not so much responsibility (the business wasn’t that big after all) and people needing you? What about starting something entirely different from scratch?

Restart, but really

This is the “make them part of the new team” from the Dungeon Master Perspective, and I said it’s very unlikely to work. But my statistical sample isn’t big enough to give certainty, and I had the privilege to meet unicorns in my profession. So in case you want to try…

… erase the past solution from your mind. Get away from the comfort zone. Everybody knows that you can solve the problem with your usual tools, you’ve done it already. The “Master” thing means that you become a DM because you’re good.

But start modelling with entirely new tools instead (well … EventStorming wasn’t probably around at that time), look for different implementations, maybe just because the original one is boring. Software development is learning, and that’s where the fun is.

After all, Sting managed to reinvent himself after The Police. ;-)

Conclusion?

As I mentioned before, frustration was the main driver for writing this post. Mostly the frustration of seeing project settlement where people are supposed to act in a way that is not going to work, and to see the struggle on both sides of the fence.

Logic and goodwill alone aren’t going to solve the problem.

Most of the annoying actions of DMs are more the result of unconscious habits than deliberate intentions, but often teams are reading deliberate evil intentions (as some of the readers did, btw). I am merely recognising the recurring situation, naming the anti-pattern, with the hope to raise some awareness, and to start finding some better solution, since ignoring the problem won’t help.

Update: this post raised some turmoil. So I guess that this is the “Clarification section”.

One thing that I didn’t intend was to raise a generational/age issue. Thanks to Ann Witbrock for pointing me to that. It’s not age: it’s attitude, systems and vicious circles, and I am 45, not the young punk somebody portrayed. It starts because somebody writes a piece of software that allows a company to grow.

This is not a technology issue too. I am using microservices and Node.js merely out of examples of unavailable options at the time the software was written. Technology can be used as an excuse but it’s usually neither part of the problem or the solution.

Possible gender bias had been a critic too. For many reasons (maybe my sample is too little or not diverse enough) I can’t recall any situation like this that involved women.

Another thing that I didn’t mean was to treat people like assets (although I was aware of the danger). Thanks to James Cauwelier for the wake up call. The reason that drove me to write was the frustration in seeing goodwill-based approaches fail, and the resulting suffering in both sides of the struggle: one of the worst tortures is to force people to act against their own interest, and helping a team to prove yourself stupid or obsolete, or simply to kill your creation, is one of these. The team isn’t having a great life either: they’re playing with a system they can’t beat; all the smart decisions have already been taken.

I also haven’t pointed out to solutions to mitigate the problem at the system level, mostly because it’s already too late. I don’t think the problem will be easily solved within the timeframe of an average project. That’s why extreme measures start to make sense.

So, this turned into a warning for developers about the risk of having an anti-pattern as their project setting. In a longer timeframe, culture definitely plays a role: freedom to experiment, a celebration of failures and learning is great to prevent this situation from emerging.

I also think this won’t be enough: I’ve seen people setting their own standards for themselves, despite a more relaxed culture around. Perfectionism is a common trait among DMs. Established practices like pair programming and collective code ownership (or whatever brings into collective learning) will help preventing the rise of DM. Many thins originate from solo programming, particularly from late night solo programming, and organisations should be aware of the risks and dangers associated with this approach.

I’ve also being accused to “vilify the role”. No, I am naming it. James suggested me not to call it an anti-pattern or to point fingers to the person.

I think the problem is the situation, not the person. But I still think the anti-pattern name should refer to the person: nobody wants to be or become the Dungeon Master. That’s the reason why I named it: early wake up call!

To tell the whole story, the vilified character is exactly how DMs are seen by the new development team. And this is the part that scares me. Then you talk to the person and discover very human behaviour, with a blindspot in seeing the consequences of some habits. Or you discover deep knowledge and the lack of skills needed to pass it to collaborators. Human things. They’re good people that will look evil in the new team’s eyes, unless we try to do something different.