Modernizing Technology and Mindset with ‘Enabling Teams’

It took us more than five years to get to this point, but we finally succeeded. We found a way to modernize the R&D technology stack and mindset to boost team productivity, delivery and satisfaction, while reducing burnout.

But CyberArk is not unique in this sense. Every company, especially successful ones, will at some point encounter inertia to change. Those shiny state-of-the-art technologies and work processes will one day become a legacy, a burden to progress.

Here’s the story of how we managed to create a sustainable flow of teams that successfully modernize their technology stack and mindset. This process significantly improves their DORA Metrics (deployment frequency, lead time for changes, etc.), which is strongly correlated with their ability to create value for our customers.

The Origins of Inertia

As a 20+-year-old company in the tech sector developing security software for enterprises, CyberArk has always been very cautious around its development process. It hasn’t been the first to jump on the new shiny technological “toy” or “move fast and break things.” The security of our customers has always been, and still is, our top priority — even if it comes at the expense of velocity.

A few years ago, reinforced by the need to build a SaaS offering, we saw an opportunity to modernize and align our services around a newer SaaS cloud-native technology stack and modern development practices, such as Continuous Delivery.

Constantly busy with their day-to-day tasks and not proficient with the new stack, the teams couldn’t justify rewriting existing code. Lack of knowledge and higher initial cost, higher risk, lack of “proven earlier successes” and lack of supporting infrastructure all provided little incentive to develop new code using the new stack.

Imagine if you were a team leader — what would it take to convince you to change the technology stack your team is using? What would it take to convince and teach your team members? You know that technology changes are akin to religious wars — think about Angular vs React, Java vs Python vs Node and many more.

So how do you convince the teams to adopt the new stack?

Jumping Headfirst

At the time, my team and I made a somewhat risky, some might even say controversial, decision to go all in with the AWS Lambda technology, AWS Cloud Development Kit (CDK) and Python as CyberArk’s primary tech stack for SaaS. What led us to choose this stack deserves its own post, so I’ll leave it at that.

As for the mindset part, we strongly believed, and still do, that each developer must be able to push to production without manual gates beyond the PR review, multiple times a day, while complying to high quality and security standards. This developer would also need to answer PagerDuty, read production logs and fix service-related production problems. You build it — you run it.

All this was great in theory, but practically no one in the organization, including my team, had experienced this in practice. This is where my team, affectionately called Neo, like in the movie The Matrix, but more officially — the Product Technology Office — had its first major breakthrough.

Back then, at a team meeting, I raised a question: Do they want to dedicate between one and two years to develop our first cloud-native service (the new SaaS Platform) and take it all the way to production, or do they prefer a more low-touch involvement?

All team members unanimously voted for being a part of that SaaS Platform team, developing business and infrastructure code and establishing best practices, both for the code and the Continuous Delivery and strong ownership-without-handoffs mindset. Everyone felt that this was the right decision. After all, how could we lead others without having experienced it ourselves?

Needless to say, I supported this decision, fully realizing that both I and the team would not be available for additional projects for the duration of this engagement.

The goal of my team for that period was to gain as much knowledge and experience in this technology and mindset as possible, while helping build a strong team that would take over the SaaS Platform development when Neo continues to new adventures.

Naturally, the Neo members were the de-facto skeleton of the newly formed team. This made the process of transitioning more complicated, as we needed to make sure the remaining team was fully capable of delivering on its own. But more on this later.

Enabling Teams in Practice

Two years have gone by, and the SaaS Platform team has been recruited and trained. They have become fully autonomous. The service hit the production environment; all the right controls and procedures were implemented. Business teams could start using the shared services.

By having completed such a massive engagement with the SaaS Platform, the Neo team gained several defining characteristics: high visibility and credibility within the whole R&D organization and proven skills required to engage with more teams on their modernization journey.

During the period of building the SaaS Platform, business teams and management started to see SaaS cloud-native opportunities in many areas. A new significant feature — an extension of the existing service or a new business initiative — became candidates for the evaluation. Are the risk and higher initial cost of developing with the new tech stack worth the potential value received?

Yet even in the face of these new opportunities and the SaaS Platform readiness, in most cases the decision was to continue with “what we have and what we know.” What drove such decisions, even at an intuitive level, was the value-cost-risk equation: the value was unclear; the cost and risk were high. It’s worth mentioning here that we are not a totalitarian organization — we believe the managers responsible for the execution need to make their decisions. Pushing unaccepted decisions down the chain would likely have alienated the teams and failed.

I wanted to make the right things easy for the decision makers, to offset the value-cost-risk equation and to onboard business teams to the new tech stack and the SaaS Platform.

The only method I found that worked was providing my personal and Neo’s support to the teams who dared to take the risk and choose the new way.

Neo team is composed of experienced and versatile software engineers and architects, adept in multiple technologies as well as CyberArk’s products. They are light on their feet and can fulfill a wide range of tasks in any team — starting with developing business and infrastructure code, CI/CD pipelines, software and system architecture, building POCs and presenting them at any forum. They are curious and not afraid of the unknown. Thus, having a Neo member on a team is a force multiplier.

In fact, Neo acts as an Enabling Team. This term was coined by the highly recommended book Team Topologies, where an Enabling Team is defined as “a team that assists other teams in adopting and modifying software as a part of a transition or learning period.”

There are two main flavors of engagement:

1. They are an integral part of a team, either part or full-time. This is a so-called Collaboration mode. They do whatever needs to be done — build architecture, develop code, evaluate tools, etc. This form of engagement is most suitable for completely new and poorly defined projects, when a lot of technological discovery needs to happen. At times, this occurs before the “real team” is assembled, as it’s difficult to define the required job description for the perfect candidates. Some modernization projects also require historical business knowledge to better understand the intricate details of the existing implementation.
2. They work in a Facilitating mode. As such, Neo does not write code or builds architecture but, rather, escorts the team. For example, they take part in the design and code reviews at the sprint preparation and design brainstorm meetings, at technical POCs and at other team ceremonies. They may also conduct custom-built training sessions for the team. But most importantly, they build a high level of trust and instill a feeling that they are here to help, so the receiving team feels welcome to ask for help, a review or a discussion.

The best thing about such a model is that the journey is kept at the personal level, where any engineer can understand and use the new technology and adopt a new mindset. This keeps the teams engaged and wanting to learn and succeed, so even when Neo moves out to other projects, the team is capable of proceeding autonomously.

Changing the Value-Cost-Risk Equation

Neo’s assistance changed the value-cost-risk equation, helping managers make the decision to pursue the modern stack, which in turn drove up demand for more modernization projects and Neo involvement.

● Driving up value: At the beginning of our cloud-native journey, the value of modernization was somewhat vague, especially for teams that use a relatively modern stack, such as Java, containers and K8s. But one of our recent projects involved one of these teams, who managed to develop a completely new service using the new stack and SaaS shared services and went to production within four months! This team also adopted the “you build it — you run it” paradigm, which eliminated handovers — previously out of reach. This topic also deserves a special post.

● Reducing cost: Having a Neo team member (or members) escorting the planning, architecture, execution and delivery significantly reduces cost. They are adept at seeing the big picture, building the right architecture and developing code. With proven experience from the SaaS Platform, Neo’s engagement with new projects would act as a force multiplier.

● Lowering risk: This is probably the most problematic part of the equation. When a manager commits to a delivery date or a team undertakes a sprint commitment, you don’t know what you don’t know. Thus, having the Neo team onboard who has already “done it” and who has proven to be a valuable asset time and again minimizes the risk. It also allows them to address the risk earlier and with greater experience.

In addition, having my personal support and when needed, additional assistance for such modernization projects, also helps the line manager undertake such risky commitments.

The Challenges of Scalability and Demand

Yet there are still some challenges. Our approach doesn’t scale well. Onboarding new members to my team proved to be a challenging task. This person needs not only to be a pioneer with great technical depth — from system architecture to cloud services to development and CI/CD pipelines — but must also be familiar with CyberArk’s services and their challenges, trusted by developers and managers, and be very service-oriented. This scaling challenge caps the length of time and the number of projects Neo can be involved in. Currently, there are around seven such projects.

On the other hand, new teams, especially those acquired non-organically, may benefit from our help but not turn to us because they lack personal acquaintance with me or the team.

Also, as time goes by, we’ll need to find ways to not only escort team modernization from legacy to modern but also encourage and maintain such “self-modernization” ability of the teams without our further assistance. In other words, we will inevitably find ourselves in situations when a team outpaces Neo and starts using newer tools, techniques and methodologies. We need to be sensitive enough to recognize such situations, learn from them and potentially use them as the source of our new best practices for other teams.

And finally, we are driven by the teams’ demand, which is hard to predict and plan long term.

Alternative VC Investment Model

In some sense, I see myself as a venture capitalist (VC) who is looking for a good investment, only instead of capital I invest my team and my personal time. So, I evaluate potential engagements and try to gauge their chances of success, the strategic importance of this project and resource availability on my side. Evaluating the chances of success is not a precise science; it is mainly a mix of the manager’s determination, the team’s engagement and capabilities and the probability that this effort is given enough time to succeed but enough business value that it doesn’t get yanked.

At times I need to pull my “investment” out prematurely because the team has not shown expected results, or the ROI of Neo’s investment has dropped. This happens if Neo is doing tasks which can be done by any other engineer (for an extended period), so the teams’ special skills are not coming into play. But more often, I increase my investment to help a struggling team.

At times we have some availability to pick up new projects; at other times we are overextended and must pass up good investment opportunities.

Where We Are Today

CyberArk R&D now counts well over 50 development teams — each with its own character, culture, product, technology stack and mindset. Some of them were “born” cloud-native, and an increasing number of them join forces with Neo and successfully manage to modernize.

I believe we’ve passed the tipping point of organizational support, when there’s increasingly less need to explain the value of the change and the process of the change. Now we are at the point when the demand exceeds the supply, and all that remains is optimizing our investment and streamlining the process even further.