Five Product Lessons From Building Columinity

Why the actual challenges of product creation are not with the idea but its execution

The past three years have been quite a ride. It gave me and Barry Overeem the opportunity to design, create, and evolve a software-as-a-service platform called the Columinity (or the Scrum Team Survey as it was formerly called). What started as a hobby project matured into an ecosystem of related tools designed to help teams drive continuous improvement with data. Today, our tool has been used by over 13.500 teams from across the globe. While most teams use the free version, we are proud of the growing number of customers — over 100, large and small — whose revenue helps us maintain and sustain development.

One of the joys of developing this product is that we’re learning a ton about what it takes to do so. While we’ve both been part of product development teams in the past, those teams were much larger. Some users believe we have a team of developers, designers, testers, and support staff. But our team is effectively just me and Barry Overeem. He writes most of the do-it-yourself content and helps with feedback while I handle all development, testing, security, hosting, and support. We occasionally hire Wim Wouters of Fonkel.io to provide visuals and UX design.

“We know that some of our users believe that we have a whole team of developers, designers, testers, and support staff. But our team is effectively just me and Barry Overeem.”

I want to share what we learned about product development in this post. In particular, I want to shed light on those areas that are part of product development but weren’t top-of-mind for us. They won’t be at the top of your mind either. So, if you’re in the product development business or hoping to do so, you might find value in our learnings. Do keep in mind that what I write below is just my own experience. I don’t mean to generalize to other teams and products at all.

A bit of background on our product

We designed the Columinity to help teams drive continuous improvement based on data. We created the tool because we know from experience how difficult this can be. Where is improvement most important? What kind of improvements make sense? How do you track improvement over time and know when to celebrate?

In short, Columinity allows you to:

• Get a sense of how well your team is doing on an evidence-based model for the effectiveness of Scrum/Agile and non-Agile teams.
• Identify the areas where improvements are most likely to increase your effectiveness.
• Use our evidence-based feedback and improvement suggestions to start improving tomorrow.
• Aggregate the results from many teams in the same organizations in our Teams Dashboard. It’s an excellent way for Agile Coaches, Scrum Masters, teams, and management to identify larger patterns in the challenges.
• Group, filter, sort, and export the results from many teams.

Over the past three years, we delivered countless increments of the product. A new version goes live at least once a week. We haven’t experienced severe bugs or outages yet and are on track with our roadmap. So the “development” part of “product development” is going well. What about the rest? What did we learn about product development that we didn’t expect?

1. Business development isn’t easy

The biggest challenge we face, which we haven’t resolved yet, is how to generate enough revenue from the tool. Although we offer many of the tools' features for free, we have running costs that need to be paid for, such as hosting, third-party licenses, insurance, penetration tests, legal and accounting expenses, development work, etc.

Most internet startups rely on outside investors to fund running costs as the product matures and grows. Because we don’t want to relinquish control of our platform, we decided against outside investors. So we self-fund instead. This makes it even more critical to develop a business out of this. Unlike many startups that can continue to make a loss well into maturity (and Spotify, Uber, Snapchat, and many others still do), we don’t have that luxury. That’s both a good thing and quite stressful at times.

We discovered—not unexpectedly—that business development is essential to product development. Unfortunately, neither Barry Overeem nor I naturally gravitate toward that kind of work. So, we asked for help from entrepreneurs in our network to generate potential ideas for generating more revenue. We received generous help from Thiery Ketz, Laurens Verwijs, Hermen Heinen, and Danny de Graaf. They helped us with three insights.

1. We should create a roadmap for the product and one for our (business) ambitions. If we aim to generate just enough income to cover operating costs, we need to reduce costs and make the product sticky to retain customers. If the aim is to sell the product down the road, it must appeal to prospective buyers. If we do something on the side, we can make it free and generate revenue from other services.
2. The primary audience for our product—teams—does not usually hold a company credit card to get a paid plan. Teams typically have to convince their managers to do so. Since this can involve lengthy procurement procedures, managers aren’t always excited by this prospect. So, we must find creative ways to help teams convince their managers or circumvent this altogether.
3. It is surprising how easily customers pay many thousands of euros for a two-day class or workshop for 16 participants while at the same time struggling to pay 150 euros for a year for a product that helps a team improve continuously throughout that year. We are convinced that the latter has a bigger impact on teamwork and behavior. But how do we help customers see this in an industry that thrives on workshops, training, and certification?

Taken together, we realize that we still need help in this area. Business development isn’t our forte. But it is a part of successful product development.

2. It is hard to feel proud of your creation

Looking back at our work in the past three years, I’m astonished. Our product has become much more than I expected. Many features that I never expected we’d get around to implementing are now part of it. Many features that I expected to be very difficult to implement are implemented fairly easily. We are actually making some money with it, though not enough.

At the same time, I feel little pride. Despite the overwhelmingly positive feedback from users and customers, all I see when I look at our product are the small bugs, the imperfections, and the unclear user interface in areas and areas that need work. I also see all the work compared to the revenue it generates and wonder if it's worth it. I suppose this is a personal limitation, part perfectionist personality and part typical for creators. I’m too close to the product to see the big picture.

3. Naming things is hard

“Naming things is one of the hardest parts in coding” is a joke among developers. It's also true for product development in general. A wonderful example is the old name of our product: Scrum Team Survey. It suggests that it's only for Scrum, which isn’t true. It suggests that it's only for teams, which isn’t true. And it suggests that it's just a survey, which isn’t true. So it's a pretty horrible name. We ended up with it because the earliest version of the product was a survey for Scrum teams. It aptly described it then. And so it stuck around.

There are other examples. We named one of our components “Team Dashboard” when it is a “TeamS Dashboard.” We also initially named another component, “Broker Dashboard,” until we discovered that “Coaching Center” made more sense. Similarly, we would’ve picked different names for parts of our model based on what we know now. “Stakeholder Concern” is a little vague. We also ended up with three types of feedback we had trouble naming: quick tips, quick actions, and do-it-yourself workshops.

The gist of all these examples is that naming your product and its parts is hard. Especially when that product is built incrementally and based on feedback from stakeholders; in that case, you don’t always know where you’ll end up. We often learned a better name for a new feature as we developed it and learned from users how they used it.

What we learned from this is that naming is exceptionally hard. It is probably also better to avoid descriptive names. While we started with “Scrum Team Survey,” the name quickly became too constraining. Fortunately, we rebranded to Columinity in late 2023 to a less agnostic name.

We really like “Columinity” as a name. We also understand that it doesn’t roll off the tongue right away. We’ve come to see it as a strength and it takes some practice to remember it and to start recognizing the elements that are embedded in it (collaboration, continuous, illumination, infinitely). But honestly, a reality is also that finding a brand name that is globally available is insanely difficult. We tried hundreds of potential names, but there were always issues; the domain name wasn’t available, the name was already used in some country, a patent with a similar name was already filed, a similar name was used elsewhere or the name had a meaning in another language that didn’t match with our intent, and so on. We even got a few letters from lawyers who felt that the name we were testing was infringing on their customers (for example: “Luminate”). This is another reality I hadn’t considered when I started this company; finding a good brand name is super hard.

4. Defining a security policy for your product is a great way to raise security awareness

I’m not a fan of policies. They feel bureaucratic to me. So, I wasn’t too excited when we had to set up a security policy to meet a requirement for cyber security insurance. However, I quickly discovered that setting up such a policy is enlightening.

A security policy outlines how you ensure your product's security and safety. Since I’d never set up such a policy, I asked ChatGPT to generate an outline I could paste into Github (in Markdown). This worked remarkably well. So, I spent the next few days filling in the details. I covered areas such as:

• In the case of a security incident, what procedures do we need to follow? How do we collect evidence? Who needs to be informed?
• What tools, software, and physical devices are relevant to our product, and how are they secured? Who has access to them?
• How have we secured our (hosting) network against intrusions?
• How do we prevent vulnerabilities in our product from being exploited? How do we scan for this? How and when do we resolve them?
• What potential incidents could occur, and what steps must be followed? For example, what if a lightning bolt hits me and can’t work anymore? Or what if hackers install malware on our servers and try to extort us?

Screenshot of our (private) Security Policy on GitHub

ChatGPT was exceptionally helpful here by offering examples, questions to answer, and even checklists. This meant that I had a structured way to review and provide details on how we secure our platform and how to deal with incidents. It also confirmed that we have our things in order in this area. So, we discovered that setting up a security policy is part of product development.

5. A lot of legal expertise is needed

Hiring legal expertise was a substantial expenditure in the past 12 months. This might be surprising for a small company like ours — it certainly surprised me — but we needed help from legal experts in several areas:

• Several enterprise customers wanted us to agree with their cyber security agreements (CSA) before procuring our tool. Although we prefer to reject such agreements out of hand, things aren’t always that simple. So, our lawyers helped us “red-flag” those agreements for clauses that needed to be changed or removed. As a testament to their expertise, the legal departments of said customers accepted all changes.
• As a company that operates in European countries, we want to comply with the European General Data Protection Regulation (GDPR). This was relatively straightforward as we store only a minimal amount of personal data. So, our lawyers reviewed the platform with us, identified where data was processed, and drafted the necessary legal documentation to show compliance. This included our privacy policy and a set of “Data Protection Agreements” (DPAs), which outline what customers are responsible for. Without such DPAs, we could be held liable even if they mishandled personal data (if you don’t have those, ensure you get them set up).
• Finally, we needed help improving our terms of use. Our earlier version was acceptable for the initial stage but needed a more robust version to support further growth. Our lawyers drafted a new version here, too. We also overhauled the flow by which users review and accept legal documents.

I can’t say writing the above makes my day. The Sprints we spent on this legal work and the functional changes didn’t always feel like the most valuable ones. Moreover, the costs for legal help often exceed those for infrastructure, marketing security, insurance, and so on—even for a small company like ours.

But we also recognized that it was necessary if we wanted to run our business professionally. We also owe this to our users and customers. So yes, this is one area of product development we hadn’t expected. However, we discovered that reviewing your product with lawyers is helpful.

Closing Words

This post reflects on what we’ve learned about product development while building Columinity. I shared five lessons that stood out for me. You might find value in them if you’re also creating a product or planning to build one.

Also, I always recommend trying out our tool with your team(s). We have an evidence-based model for Scrum/Agile teams and a (beta) model for other kinds of teams. So give it a try and let us know what you think! Check it out at https://columinity.com.

See how you can support us, at https://patreon.com/liberators.