Why can’t we just talk to each other?
Some of the most common questions that I seem to get frequently are about user stories: Why do they take so long to write? How do they become so big? It seems that user stories have evolved into mini-waterfall documents that don’t fit within the boundaries of a pre-defined iteration.
The problem is so common that I asked Martin Fowler, one of the original signatories of the Agile Manifesto, what the heck is going on?
His response: “The original intent of user stories has been lost. They were designed to be small chunks of work written on index cards with just enough information to get people talking and working together to solve problems. But somehow, over time, user stories have become so big and so complex that they slow teams down.”
The underlying message is that documentation should never replace human conversations. You should trust your people to work things out, trust their decisions, trust the process, and trust that people have the courage to do what is right.
Ineffective Communication Causes Bloat
Let me introduce you to a tech company… let’s call them DataGenius. This is a small American company who sells data. Their core business model is providing clients with predictive data intelligence.
In order to grapple this work, there needs to be a number of very smart people on the ground. The team has a healthy combination of roles: design, analysis, quality, developers, specialists, and management. Their iterative ceremonies are designed around 2-week iterations.
However, this company has difficulty getting software out quickly. Their product vision is haphazard and the overall direction changes almost daily.
User stories are long and difficult to read. Business Analysts and Designers are constantly “rewriting” stories so they are current and reflect newly made decisions.
Their product backlog is stagnant and hadn’t been touched in over 6-months. Every iteration has a period of stress where the team struggles to get stories ready for development. And worst of all, it is really apparent that everyone is unhappy.
One thing that comes up in every single retrospective at DataGenius is: “Our communication is broken. No one communicates with each other. We need more documentation in the user stories.”
But…. I’m going to challenge this.
I’ve yet to meet a company that isn’t communicating. I saw evidence at DataGenius of over communication. And, every time I heard them say “communication is broken,” “no one is communicating,” and “we need more documentation.” I think it was secret code for “the team is communicating the wrong things, at the wrong times, and with the wrong people.”
Typically in a “communication is broken” company, all of the talking is happening at the point of story writing, and we saw this with DataGenius.
A few days before the iteration, Business Analysts frantically rush around to capture as much information as they can in a user story and at the same time try and figure out what they are building and the direction they are heading in. The outcome is work that is not fully flushed out and there is confusing product direction.
Directional decisions and communication isn’t meant do be done at the last minute, it’s meant to be spread out over the course of product development. To have discussions at the point of story writing doesn’t give enough time to effectively think through the problem you are trying to solve, the value you are providing, your priorities, whether you are using the right tools, and what it is that you are are actually building.
Anyone can build software, but are you building valuable software… and what does value mean to your organization?
The Three Phases of Communication
Communication Phase 1: Ideas
Ideas can come from anywhere but typically they come from leadership, product owners, market signals, sales folks, and your end users. The input that each of these has can vary from company to company and via different stages of software development (e.g. new product needs vs. legacy product needs).
During this communication phase, everyone who touches the product has ideas on what to build. Research is done in the market, analytics are used to validate this research, and users give feedback on your ideas (to validate assumptions).
When you design the software platform, you think about things like your NFRs, the hardware used, the architecture, lots of other technical bits. On a more mature product, you find bugs in the software, you get ideas from your own people and feature requests from your clients.
All of these are “ideas” that may or may not go into your product. The gathering of these requires communication.
Communication Phase 2: Synthesizing Insights from Ideas
Once you have all these “ideas” you then have to sort through them. This is an important communication phase where everyone on the team works together to determine the best path.
Not all of these ideas may go into a product, but after discussing and sifting through them, the team should come up with some insight as to the best way to move forward. Communication here should be exploratory and collaborative. You should continually ask, what problem are we trying to solve?
One common anti-pattern is to simply add features to software without developing insights because “a client asked for it.” Clients know what they want, but their needs may be selfish or fit edge cases that aren’t relevant to the rest of the market. A product team must balance the needs of clients with the potential of the market and not get too bogged down by features that don’t provide value.
Communication Phase 3: Implementing Insights
This is where the fine grained product planning, backlog building, prioritization, and user story writing happens. This is funnelling your work through the product pipeline to get the most meaningful, valuable work ready for developers.
DataGenius did most of their communicating and product work at the point of writing “user stories” without having all the valuable conversations upstream. This created stress, confusion, incomplete stories, and a product that was built based on “what could be done now” and not “what has the most value.” And, in a lot of cases, developers picked up the analysis slack and defined product direction.
When this was identified at DataGenius, they began to stretch out and separate their conversations so that important decisions were made earlier so preparing for the next iteration became less stressful. When it came time to write user stories, it was a quick activity that captured important details. At the point of development, everyone knew what they were supposed to do because conversations happened upstream.
This Needs to be Iterative
This entire process should be done iteratively. There should never be a massive runway of conversations, analysis, and document writing… because that’s not effective either.
But teams do need breathing room to work, discuss, and communicate in a healthy way. This should be ongoing, iterative, and full of trust and courage.
The farther away from development, longer the stretches of conversation are needed. The closer you get to development, the shorter the conversations should be because you’ve already worked out the value, the details, and the knowledge needed to proceed.
Teams need to understand that communication shouldn’t be done at the last possible moment. There should be just enough iterative urgency to keep people focused, but not so much that you’re rushed and the product is not well defined.
Some Key Take Aways
Communication involves everyone on the team at the right moments. It’s up to the team to identify when those right moments are and be aware of when it’s “too late” or “too early” in the product lifecycle.
Remind yourself that user stories are a reminder of conversations that have already happened and are not meant to be detailed upfront documentation to be followed explicitly. They should describe the work being done, provide valuable specifics of that work, and contain a definition of done so the work doesn’t drag on. If questions still exist, conversations need to happen. Stories do not replace human interactions.
One of the most common assumptions that people make in using Agile practices is that you can make important decisions at the last possible moment. This is untrue. Agile is about being fluid in handling change and using practices allow you to quickly pivot when you learn new things; but, Agile is also about identifying the last responsible moment for this change. It should never be used as a tool for deferring decisions about overall product direction. Value still needs to be determined before you get to the point of story writing so you know you are working on the right things.
And, one final note, one of the big blockers to effective communication is courage. It takes courage and trust to communicate effectively at all levels in an organization. For some it’s very easy to work around a problem in order to avoid talking to someone you don’t like or are afraid of, but a simple conversation could save hours of work and rework. Be brave.
The Outcome for DataGenius
With DataGenius, we focused on building out the product design function and moving these conversations earlier in the product lifecycle.
We focused on involving designers earlier in the product cycle and focused on improving the collaboration between design and analyst teams. We helped everyone understand that “design” isn’t simply providing mocks for user stories; but rather, incorporates elements of usability, interaction, and overall product vision.
We involved all levels of technical leadership in early conversations and quickly pulled back work that wasn’t ready or feasible for the team given the current state of the software. We made sure this happened before work got into the hands of developers.
We set up a product wall that showed ideas, a prioritized backlog, and all the stages of analysis; we talked about these at a daily product standup. Designers and BAs worked together to continually prioritize their work and push it incrementally through the analysis lifecycle. By the time user stories were ready for development, they were fully flushed out and easily picked up by developers.
We focused on improving overall team communication and encouraged people to talk to each other rather than document in user stories. We did this through feedback sessions and by continually asking, have you talked to that person? We looked for people who were hiding or who were avoiding conversations and focused on building relationships between individual team members. Ironically, this was the hardest task to accomplish because it involved removing a lot of baggage.
Finally, we helped the team understand that the reason why they couldn’t get software out quickly wasn’t because of a “development problem” or a “documentation problem,” it was because of a lack of effective communication at the right moments.
The outcome: the team blossomed and the work environment became lively. Development sped up and the team had insight into what was coming down the pipe so they could plan for it. Retrospectives stopped focusing on the company’s “communication problem” and discussions revolved around creating value, working on the “right things,” and paring down the documentation ceremonies to increase speed.
