Challenges of Transformational Change to Become a Software Company

It’s hard to believe that software is even more critical to business success than a few years ago, but it’s a reality that the livelihood of many historically non-technical corporations now depends on growing a software development competency that is the foothold of their competitive advantage and/or their ability to execute — like it or not. Ready or not.

Over the years, I’ve seen and helped drive new technology and approach adoption, typically hired into organizations that did not have requisite expertise in-house; serving as a catalyst for change, an arbitrator, and necessarily an internal educator. The fundamental software transformations of the last decade or so have included for me: Service Oriented Architecture, Agile, REST, Microservices, and the Cloud. Take a look at A Decade of Evolution in Architecting Reusable Software for a synopsis of the technical evolution driven by these transformations.

Having lived through multiple waves of technology adoption while working at a diverse set of organizations has been an eye-opening experience, giving me an appreciation for the challenges that non-technical organizations face to become software companies. The largest and most likely impediments to transformation are shared below. While this story is anecdotal in nature, trends observed across transformational initiatives over the years provide some substantiation for the patterns of challenge discussed.

Lack of Organizational Adaptability

The single-most important factor of transformational success is if change is a key competency of the organization where this is not only an explicitly stated precept but also pervasively supported and pursued across the enterprise. If it is, the organization has likely got several transformations under its belt and is poised for the next wave of change. If it’s not, then people, process, and tooling are likely not up to the task of change, or at least of changing efficiently.

A key quality of an organization is its technical adaptability — a measure of how readily it can adopt new technologies and technical approaches for competitive advantage. Organizations born in the software industry are accustomed to and able to perform in cadence with the Silicon Valley pace of innovation. For corporations used to competing in slower industries, it can be quite a challenge for them to keep up. Moore’s Technology Adoption Curve provides a good framework for categorizing an organization’s adaptability.

Moore’s Technology Adoption Curve as Explained by Everett Rogers in “The Diffusion of Innovations”

Except for the most risk-sensitive industries, it’s my assertion that any sizable organization should evolve to the point where they are generally Early Adopters with strategically chosen capability domains where they are Innovators. Consistently and correctly selecting when to be an early adopter is a great indication of organizational maturity and the ability to adapt competently.

Organizations struggling to adapt have a tendency to be insular, that is, they predominantly look inside and not outside for solutions. Unaware of industry offerings until late in the game, innovations eventually get brought in-house and can be touted as novel when for industry they are not. From personal experience, a 5 year lag in the uptake of new technologies is not uncommon.

An organization on the path to being a competitive software provider must instill in its culture the value of adaptability, cultivating it across all facets of the business. An intentional culture of adaptability provides the framework for transformation. Never is this more important to follow than when recruiting, developing, and retaining employees.

Employee Mismatch with the Emerging Organization

An organization embarking on the journey to become a software company is likely not composed of the right type of employees. The more pervasive this problem, the more difficult the transformation journey. From past experience, the alignment of the employee with the emerging organization can be categorized by employee state:

  • Executing. The employee exhibits behaviors that are a cultural fit with and has the skills needed to be a contributing member of the emerging organization. The employee is the future of the organization and is therefore a good investment. The employee directly or indirectly helps facilitate organizational transformation.
  • Escaping. The employee has the requisite skills and personal qualities needed to be a contributing member of the emerging organization but is not executing inline with that potential due to environmental blockers. Environmental blockers exist because the current day organization is on the path to becoming its new self but is not yet there — elements of its legacy inhibit the employee. These blockers can take many forms, including such things as having a manager that is not a good fit with the emerging organization or antiquated processes. Without intervention, the employee will eventually leave the organization. This is a common problem for new employees hired specifically to accelerate organizational transformation that have to live in the early days of this change journey. It is imperative for the organization to identify these employees and take actions to retain them.
  • Awaiting. The employee does not offer the skills and/or behaviors that are a cultural fit with the emerging organization but has the potential to acquire and become what is needed if given training and mentoring. Ideally, the organization is full of employees that can readily adapt and change but practically speaking, unless the historical organizational culture championed adaptability, it is a lot more likely that a non-trivial percentage of the employee pool will not be able to make the transition that the organization requires. Past experience suggests that organizations in need of hiring leaders from outside the organization to really drive change are typically going to find that a dissatisfyingly small number of employees fall into the “awaiting” category. That is just as much a failing of the organization as it is of the employee for not owning their employability. The adept organization and its employees continuously evolve, reducing the chance that very big transformations are needed.
  • Separating. Employees not onboard or unable to adapt with the emerging organization grow increasingly out of touch with its culture and practices. The employee gradually separates socially from the organization and typically his/her skills age out. The employee likely realizes the emerging organization has needs he/she can’t meet. Separating employees can fight to maintain their organizational position but this typically only defers the inevitable — separation from the organization. An employee may also have a lack of interest in the future organization and proactively leave.

Given that employees are the lifeblood of change, it is important for an organization to know the personal qualities of their next generation employees and to seek out such people. Without being overly prescriptive, successful software companies for the modern era will have employees that are:

  • Technologists. Good software companies are filled with people that have a passion for software, proactively engaged and thriving in an ever-evolving industry constantly abuzz of the latest wave of technology. These folks are likely doing personal projects or consulting gigs on the side, remain in the know on technology trends, and generally bring a lot of enthusiasm for technology to organizations.
  • Innovative. Good software companies are filled with inventors that offer a combination of software development know-how and creativity. They increase the chance of an organization bringing new products to market that are disruptive.
  • Specialists. Organizations that start off as non-software companies have subject matter expertise in the domains of the original business but not in the domain of software development. This is in stark contrast to an organization that starts off as a software development company that has hired and expects software specialists to fill its positions. Transformation naturally includes greater software specialization and the sometimes awkward change out of key personnel that lack the requisite skill sets as the expertise bar is raised. An organization will not grow its competency and product suite correctly without this change out.
  • Change Agents. Building a culture of organizational adaptability is a lot more likely to take hold if many of its employees are naturally malleable. There are some people that need change and are easily identified by looking at the amount of variety in their personal and/or professional lives. Still others change easily. These two types of people indirectly lead change by modeling adaptation behaviors for others to mimic. Still others are natural influencers, making great change leaders.
  • Collaborative. To be successful, developing software that solves complex problems requires teams of people working together over long periods of time. Employees that can readily get along with colleagues and know how to solve problems as a group tend to produce more value for the organization.
  • Customer Focused. Employees that are naturally motivated by helping and have empathy for customers are better contributors because this caring translates into doing the right thing for the customer even when this may be inconvenient.

Immature Software Engineering Practices

For non-trivial sized organizations, robust software engineering practices and their systematic application to deliver software products consistently requires years of maturation. Each wave of technology advancement necessitates software engineering practice improvement.

Experience suggests that even before building out practices for new technology advancements, organizations typically must first strengthen their existing software practices. For example, in the days of SOA, organizations I worked with did not have strong enough design practices to build out the SOA variations of those practices. For an organization becoming a software company, the gaps of existing practices can be non-trivial inhibitors that must be identified when coming-up with a realistic improvement plan.

With a plan in place, an organization becoming a software company must diligently mature its software engineering practice, remediating gaps and obtaining legitimate mastery of the new practices of the day. For example, if the organization is rolling out an Agile practice, it needs to doggedly pursue competency until it is excellent. This is very much the organization becoming a software specialist at the enterprise level, which is easier said than done. Lacking a baseline of mature software engineering practices coupled with not having an employee pool with technical depth can cause the transforming organization to not master the practices of the day, disadvantaging the organization for the next wave of practice adoption. Imagine the problems that would result if building a house from the ground up where first the property lot is only partially prepared, and then the foundation is built only half right, and then the flooring is added but not up to code, etc. Under analogous circumstances, the organization becoming a software company will not likely resemble best-in-industry competitors and pervasive organizational problems will likely drive problems into developed products.

For the transforming organization, there are no shortcuts to maturation. The organization must plot a realistic course and achieve excellence at each step of its journey.

Substandard Software Development Infrastructure

Just as with software engineering practices, the development infrastructure to build and run software is established over time as a software organization matures. Industry leading software companies have a rich set of tools and runtimes. From a continuous integration and continuous delivery pipeline to reusable libraries and everything in between, engineers are enabled to build software products as this infrastructure makes them efficient and the development process more enjoyable. Organizations becoming software companies will likely have to make an investment in infrastructure for several years to improve development execution. There seems to be a tendency for leaders not coming from a technology background to have difficulty accepting the necessity and/or size of this investment. Leaders growing-up in software companies readily understand that the value proposition, though indirect, is nonetheless strong. Good infrastructure makes software engineers efficient, reducing the friction of getting work done so that products inevitably get released faster. Just as important, it keeps software engineers happy and that has numerous benefits, including the retention of top talent.

Lack of Innovation

An organization can survive a long time by just developing the capacity to readily adopt new technologies and technical approaches that others in industry pioneer. This is the first bar that a non-technical organization should attempt to achieve and one that rewards nicely. This necessarily requires active participation in industry including conferences, standards bodies, online forums, and industry consortiums.

When you are a company that does not yet regularly produce industry-leading innovation but instead largely follow industry evolution, it indicates lack of engineering excellence, implicating engineer quality and likely quantity as well.

When you are a company that does not yet regularly produce industry-leading innovation but instead largely follow industry evolution, care must be taken to avoid ill-advised technology adoption. An organization transforming to a software company may not have the expertise to separate hype from new technologies that can deliver business value as purported — causing a lemming type following effect where the organization tends to adopt everything that is new without proper vetting. The organization needs to become a discerning adopter where employees can critically evaluate technologies, filtering out those that are not technically sound as well as those that are not a good fit for the specific needs of the business. This typically involves a shift from passive adoption after-the-fact to proactively seeking out new technologies that are a good fit for problems the business needs to solve.

Employees that are used to following and not innovating technology may feel uncomfortable with novel solution proposals, not trusting internal innovations due to an over-reliance on industry players telling them what is right. This will slow down the journey to becoming an innovative company and may frustrate newly acquired employees brought in to help transform the organization.

Inevitably, the transforming organization needs to raise the bar and become consistently innovative. There is a lot to say on how to achieve this but it starts with hiring technically creative people, establishing an organizational culture that supports risk taking, and intentionally providing opportunities for innovation to occur. These are yet more areas that necessitate transformation where challenges must be overcome.

The Reality

It’s not really that easy for an organization to become a software company if that is not its roots. Some degree of transformation is certainly possible and may be sufficient to compete in industry. Can an organization transform to the point that it is indistinguishable from an organization that started out as a software company? From what I have seen so far, no. Perhaps that level of transformation requires many years in the making.