How a long feedback loop hurts your R&D

I’ve been a part of the medical software world for a number of years now, and let me tell you: Developing software for a medical company is difficult. You have a lot of regulations to contend with. You must satisfy numerous parties before your software will be used anywhere. This invariable truth will mean that you cannot avoid a longer feedback loop.

But I’m getting ahead of myself. First, what is a feedback loop?

It is the time it takes from the moment you make “a thing” until the moment you have got any response on how well that “thing” is doing. I’m being ambiguous since this loop comes in many forms, and it is different for different stakeholders.

For an investor, it might be between making the investment, and getting a sense on how it is doing, or actually seeing revenues.

For a product owner who defines the features of a product, it will be from creating the definition of the feature to getting user feedback on whether it works and provides important functionality.

For a developer, it will usually be from writing the code to knowing whether it is working properly. But this is another subject that we’ll expand on later, since the end goal can mean various things.

But what is a long or short feedback loop? What length of time is considered “long”?

Again, this is subjective to the context of the stakeholder. We’re here to discuss the impact on R&D, so let us focus on that context for now.

Knowing your code works

Say a developer writes code and runs a test that was designed to verify whether the code works in a way that matches the requirements, one could say:

If your tests pass — you know your code works.

Unfortunately, software is more than straightforward code. It needs to be compiled, packaged, deployed somewhere (probably a server in the client facility or maybe the cloud), installed on a client-facing device (like from the Google Play store), and run by the user.

All of those are technical aspects. There are also the dreaded non-technical aspects. Are the users using the software as expected? Are they installing it on devices it’s not meant for, like installing a phone app on a tablet?

If I wrote a function and the tests to check it, then I have a very short feedback loop for that scale. Within seconds I will know if it works or not. Without tests, I would need to manually test the feature, and the said function will be a small part of it. It will take me at least minutes to an hour to check, making the feedback loop longer by a factor of 100 at the least.

The impact of a long feedback loop in this context is straightforward: it will take the developer longer to complete even a simple assignment. It might still be “just an hour”, but how much better would it be if it was “a couple of minutes”?

Knowing your system works

Say a developer writes the code for a feature and successfully tests it locally. Now they go on to test it in the deployed environment and find that it isn’t working. It should work because it’s not a big change. Further examination shows another developer is also using the same environment and is making changes to another part.

They wait a while, maybe do something else in the meantime. Finally they can try again, and it works! Great.

Job done? Not yet.

Two months later, when the system is released, reports on issues start to roll in, and apparently, the feature is not working as expected. Now, the developer tries to recall what the feature was, what the implementation was, and why was it done like that. Then they must reproduce the error and figure out the root cause. Suddenly they realize that the local development environment may match the technical layout of the client environment, but the data being pushed into it is just a tiny bit different then what they were aware of.

As you can see, in this case, it took an entire release cycle before the developers became aware of the difference between the client’s environment and theirs.

In this context, the long feedback loop is due to the testing system not exactly matching the client’s system. Not only was the length of the loop so long that the developer lost the context of the feature and took longer to return to it, but the client experienced a defect, which can have a negative effect on customer satisfaction and developer reputation.

While not every software project will face the challenge of developing a system that must match the client’s environment, it is nonetheless a very real problem that must be taken into consideration.

Knowing your UX works

Say you’re planning a new feature for your product. After tireless efforts, your UX designer creates a streamlined UI, filled with the latest concepts of flow, retention, and so on. You build on that and spend a month drawing requirements and implementing and testing the code for the new feature. Finally, you release the feature and it reaches the hands of the users. But nothing happens. The users are not really interacting with the new feature. Most of them find the UI confusing and just ignore it. No defects are reported and everything seems fine.

If the company stays passive, the feedback loop here is falsely indicating that all is fine!

Let’s roll back the clock a little and take a different approach. Say that before development, you conducted surveys and focus groups to assert that the UI design is good.

This shortened your feedback loop and saved you a lot of time.

Now let’s fast forward. You release the version with the new feature, and after a while, you see that users are not using the new feature to completion. Your logs are showing some interaction with it but users don’t reach the end of the funnel (e.g., the cart is not reaching the checkout, the form is not being submitted, etc.).

Even though we had a short feedback loop in the beginning of the project, we “lost” it.

How do we shorten this loop? Tools like Google Analytics give you real time feedback on how many people are interacting with your system. Create documentation of each session’s interactions — what the users clicked on, which pages they navigated to, and so on.

How a long feedback loops hurts your financial bottom line

In the end, all these long feedback loops mean development takes longer to release, defects take longer to fix, system installations take longer (where applicable), less users interact with the product, and so on and on.

You end up having development cost more, and with less users, revenue is lower than expected.

Whether you are a manager, owner, developer, UX designer, or any other position, during the entire project the thought should always be there, in the back of your mind: How will I know this works?