Great Demo, Great POC, Great Deal... But Why Does It Not Work?

Co-Authored with my dear friend and colleague, Tom Schryver. Inspired by real life events, but all companies and persons mentioned in the story are fictional.

1

London, UK

It was a cold Sunday evening in January 2016 when the top executives of Operato — a cloud based quote-to-cash software company in the UK — convened an emergency meeting.

The background was somber. By a weird quirk of coincidence, three of their largest customers were threatening to cancel their projects at around the same time. A perfect storm was brewing and it needed to be addressed now.

There were two common themes across these customers. One, they were substantially larger than the typical Operato customer, each with greater than $5B in annual revenues. Two, they were all clamoring for the capability to deliver dynamic and personalized user experiences for different audiences based on product, interaction type, channel, customer segment, past behaviors, contracts, and an increasing number of factors. These requirements were not new, and had come up in the sales cycles. But Operato sales consultants were masters at presenting demos and POCs, and had successfully positioned these as “easily addressable” through configuration.

Discovery, Analysis, Requirements, Proofs of Concept and signoffs all along the way did not save Operato from their customers’ now-paid-for-conclusions: Operato’s products are simply not robust enough to meet their needs, and Operato is not agile enough to respond to them.

Operato felt it may have one last chance to turn the situation around. If they could deliver the enhancements required to address the gaps, and do so in short order, they could dig themselves out of this hole.

The conversations grew animated in the meeting, with emotions running high. “This is not how we built our product,” the VP of Engineering repeatedly said. “These capabilities push us way beyond our comfort zone. Yes, we can demo these nicely in a sales cycle. Yes, we can check the box. But, no, we really cannot deploy them in production at scale. To truly meet these requirements, we need to fundamentally transform the way the product works, especially how we navigate the user, render the user interface, and the core constructs that govern extensibility.”

He took another long pause and continued in a grim voice: “At least one of our CPQ engines will need to be re-written to scale, and we hope it can just be refactored without having to touch the algorithms. I’ve already assumed that you’re willing to throw hardware at it. All of these are heavy-duty things to do with massive impacts everywhere.”

When the VP of Engineering shared his development estimates for these capabilities, the CEO — let us call her Paula — almost choked on her sandwich.

“This is degenerating into a disaster”, Paula thought. “Even with the entire engineering team focused on these capabilities, it would take far too long to rewire the internals and get this done. Can we re-negotiate what “done” means, get out, and still declare victory? How about those Systems Integrators (SIs) that we’ve wined and dined for the past few months? They’re already in play, maybe we can make them step up to do more.”

Paula was an external hire, but she had always believed that Operato was the best SaaS product in the space. They had deployed to several hundred customers in SMB and SME, and had just begun the expansion to “large enterprise”. It was all going great, until these large customers had started implementing the product.

Paula distinctly remembered how one of the customers, Mike Allen of Thurman Manufacturing, had been uncharacteristically blunt last week. He had said, “You guys really do not have a platform. You have some cool stuff, but it is not flexible or extensible. How do you really expect to step into a company as big as ours and solve our problems?”

When she had pushed back, Mike had continued in his baritone voice: “ Paula, when you say focus only on the top products and their most frequent behaviors, how do I not represent the rest of our offerings? How do I not represent the many behaviors that are necessary to sell today? It cannot be partially complete. It does not work that way. Leaving out just 1% of the $10B we sell means taking a revenue hit of $100M. It’s not even a conversation that I can have.”

Paula and her board were eyeing an IPO or a M&A in 12 months and the last thing she needed was bad PR or a miss in Operato’s revenue. Her team was smart and they had a killer product. Or so she had thought. Until now.

It was clear to Paula that none of these customers would be willing to wait for Operato to figure this out. The customers currently had billions of dollars of revenues flowing through their legacy systems, the systems that they contracted with Operato to replace. No one was going to take a hit to revenues and then still wait for Operato’s futures.

Could she control the damage by limiting fallouts in the press and recasting these projects as exploratory learning opportunities with her board? “The most important thing,” Paula told herself, “is to not let this situation devolve into something that could hurt her own and Operato’s reputation. And it really cannot impact the timing and valuation of our planned liquidation event through a M&A or IPO.”

But her instincts were telling her that this could just be the start of the end. It could become a quagmire with no good exit, and a long nightmare.The board was on a very different page in terms of growth and EBITDA expectations, and this would come as an unexpected shock that could change everything.

Her mind was racing to understand if there was a way out, that would not require her to update her board on the developments. She made a mental note to ask her legal team to review the customer contracts and understand their exit scenarios, especially around re-negotiation and potential hand-off to SIs. After all, experts on both sides all agreed on this, followed their best practices, and signed off all along the way. It can’t all be on Operato, and no customer wants bad news publicized.

2

St.Louis, Missouri, USA

Four thousand miles west of London, Mike Allen, the VP of Customer Experience of Thurman Manufacturing, was growing agitated. He had been huddled in a meeting all Sunday afternoon with his team, but despite intense discussions, they were no closer to figuring out the way forward for their next generation order management program. “This situation is truly f***** up”, Mike muttered to himself, and he could not but retrace the journey that had led to this moment.

The IT organization, working closely with Mike’s team, had selected Operato as the software vendor for the next generation order management project after a lengthy evaluation process. Multiple SaaS vendors were put through a rigorous POC over 60 days and there was no question that Operato came out on top. They delivered a great demo, addressed all the questions, and genuinely seemed like a great partner for the project. Mike and the CIO liked Paula, the CEO of Operato, and felt that they could work with her.

IT negotiated what they thought was a great deal, and the business couldn’t wait to get started. Thurman had close to $10 B in revenue, and the new system would help capture clean orders across all products and customer segments, transcending multiple direct and indirect channels. Even the CEO was excited when briefed on the program. This was a visionary project, he was told, that would catapult Thurman into the leading edge of digital transformation among its peers.

But things started falling apart just a few months into the project. Mike’s team described six broad areas of challenges:

1. Extensibility and Custom Development: It became painfully obvious that the process of modifying existing Operato capabilities and adding new ones was highly cumbersome; it required intensive coding with huge implications on reusability, performance, maintenance and future upgradeability. Now, they didn’t expect the product to be able to do everything out of the box, but they did not expect that their additional needs would require time and money which dwarfed their initial investments.
2. Open APIs: The product’s business logic was accessible within the native app, but the APIs available were very limited. This severely impacted a range of business use cases where third party applications within the larger ecosystem would need to call Operato services. One of the primary business reasons for initiating the next generation order management project was to support Thurman offerings through other channels. It no longer appeared feasible without open APIs.
3. Custom UI / UX: Operato simply did not support the basic primitives to deliver all the dynamic and variable user journeys. One of Mike’s team members described it thus: “We need a UX for the novice AND the expert. Why am I negotiating with Operato implementers about this at every step? A dozen times a day the SI threatens me with change orders for gangs of these. ‘How much additional money do you really have?’ does not win favor with a customer.”
4. Integration: The order capture and management process at Thurman had to inter-operate with a wide range of legacy and third party systems. There was no elegant way to model these data sources and data transformations, and it appeared that they would need to hand-code almost everything. Thurman’s IT Architect had made the following comment: “We believed Operato when they said model-centric implementations are 5–10x more cost effective than code-centric implementations. Now that it is clear they are code-centric, how much money do we really have to spend?”
5. Creation and Maintenance of Offerings: The Operato constructs available to model Thurman’s offerings — products, services, bundles, options, pricing and rules — and their behaviors were limited. The more worrisome part was that the features seemed hard-coded; even Operato consultants did not have the ability to change them. The demos and POCs they had seen in the sales cycle were the art of clever pre-sales “solution engineers”, but wouldn’t work with the variety and volumes of Thurman’s offerings. Thurman was taking a major step backwards relative to the capabilities of the current legacy system. Mike’s direct report responsible for enterprise product management had told him amidst laughter in the room: “It’s never a good sign when you describe your product lifecycle to your vendor, and they tell you that your PDM or ERP suites should handle it.”
6. Vendor Approach to Customer Success: Operato’s customer success people believed that Thurman was over-complicating the project. If Thurman would listen to their best practices, they said, they would be in a much better position to succeed. They were also of the opinion that Operato’s primary business was selling software, and seemed conflicted about if and how Operato would take accountability for what they believed was a highly custom project.

As the months had worn on, it was obvious to Mike that they would not be replacing their legacy mission-critical applications any time soon. Mike now realized that Operato was built for a SMB / SME environment where the emphasis was on implementing the out-of-the-box product. It was clearly not built for a large enterprise like Thurman.

But how could they have been so wrong during the evaluation? How did they allow themselves to waste millions of dollars and more than a year in an endeavor that looked almost foolish in retrospect? Mike also didn’t expect Operato to be in the dark as to the need for these additional capabilities, or how to address them. It was not malfeasance; Operato truly didn’t know, and neither did Thurman.Not only did they buy the wrong product, they bought from the wrong company.

Mike knew that they would have to prove that the new system works — at a minimum, doesn’t hurt experiences or revenues vs. the existing — before it can replace the legacy. “That by itself is significant work,” he thought, “and my vendor isn’t interested in any part of it.”

Mike was indeed wrestling with some very basic questions:

• Who is responsible for the replacement of his legacy mission-critical systems?
• How do they ensure that they don’t take a revenue hit when they move to the new system?
• Why are they “implementing the vendor’s software using best practices?” vs.“designing great customer experiences and proving how good they are”?
• Why couldn’t fundamental design choices be tested early and often? I’m about to be known as the guy who bought the ugly one. What’s in the SOW that protects me?

Mike knew he was unlikely to find joy in the SOW. He was scheduled to meet with the CEO in a couple of days and he badly needed to figure out his story and approach by then.It was getting incredibly murky and Mike knew his job was on the line.

3

The Vendor Perspective

Operato’s and Thurman’s situations are not unique and embody themes we see far too often.

It is one thing to solve a problem for small companies. And it is quite another when you sell to large enterprises.

It is like trying to play in the MLB after stints in the little leagues.

It’s because the amount of variation that needs to be supported in a full enterprise implementation is 10–1000x more than in a point solution. Digital disruption expands the scope to the entire ecosystem and causes even more variation. Not only is this amount of variation much larger, but the rate of change of everything that makes up an offering (products,component parts, marketing info, rules, behaviors, pricing, lineage, …) is accelerating as well.

The biggest driver is the expectation of the customer: “I want it my way, now, and if it’s not easy or obvious, I’ll go elsewhere.”

Designs which can handle 10 variations are very different than designs which can handle 100 variations; again different for handling 1000 variations, and still different for 10,000 and 100,000. Scaling has to handle the expected variations, the rates of change, the many user types, the many users, and not insignificant amounts of data … in mission-critical applications where everything is built for change. Very few POCs and their scoring mechanisms look at this meaningfully. Irrespective of the complexity underlying the design, the end-user experience must be fast and easy for all users across devices.

Vendors who have built functional but closed applications often do not succeed in large enterprises, outside of narrow vertical segments or use cases. Vendors who sell open toolsets, without sufficient abstraction, and reference applications, have also not succeeded in selling beyond old-fashioned IT organizations.

Hitting the sweet spot requires a judicious combination of platform infrastructure, application infrastructure and highly customizable, yet artfully assembled reference applications. The platform and its core primitives are as important, if not more, than the delivered reference applications.

Vendor marketing often conflates platform and applications, but they are distinct and unique.

Applications represent the final products used by administrators and end-users. The platform represents the declarative framework, repository and toolkit used by delivery, engineering and third party developer teams to build applications.

Most cloud vendors are not big enough to create their own platforms. For 95% of vendors, it is important to evaluate and select a platform as a service (PaaS) that is optimized for your specific domain (like Force, Azure, Cloud Foundry, etc.).

As a vendor, if you decide to build or deploy a cloud application in laissez faire mode, without a deliberate platform strategy, it may not work beyond the bounds of the initial use cases.

Even if you are selling only to smaller customers now, take care to design for abstraction, configurability and extendibility. An application infrastructure on top of the PaaS may allow you to provide this layering and texturing.

Over time, as more capabilities shift to the platform and application infrastructure layers, the end-user applications become more declarative, and easier to compose, assemble and release. This will dramatically accelerate the pace of innovation.

Achieving such clear separation is hard work and needs strong engineering chops and discipline. However, it is also a critical pre-requisite to grow the business beyond your initial markets, and to build a repeatable, scalable business in the long-term that can meet the needs of demanding customers without diluting gross margins or hurting customer satisfaction.

Great SaaS end-user experiences vs. openness is a false choice. So is speed of launching new features vs. configurability. You can and should do both.

Else be prepared to face the gut-wrenching moments that are sure to come down the pike. Just like Operato realized that cold morning in London.

4

The Enterprise Buyer’s Perspective

Vendors often deliver great demos and POCs, and satisfy the handful of the most common use cases to get the sale. But once the enterprise starts to use it, the deficiencies become very real and immediate. How can the enterprise not fall into this trap? How can they get smarter about understanding what they really need, and who can best serve these needs as a strategic partner?

The focus of the enterprise during evaluation and POCs has to be as much, if not more, on the platform underlying the product. It also has to rigorously assess the vendor’s business and operating models, and their specific contracts, to ferret out the risks, structure the rewards, and understand how they may react in various what-if scenarios.

Here are some critical questions for large enterprises evaluating SaaS vendors for mission-critical business processes. It is by no means a complete list, but just a starting point for an honest and ongoing conversation.

1. Is the application built on a robust cloud development platform?
2. Can they show evidence that the applications are not hard-coded, and that they are declaratively defined in metadata? It’s not “if” they can be modified, but show how they are modified, for relevant scenarios.
3. Is the underlying business logic well exposed as APIs?
4. Is the business logic itself implemented as workflows that leverage foundational services, so that these can be modified as required?
5. How many of the necessary right primitives and libraries — specific to the problem domain — does the platform support that facilitate rapid development and integration to other services?
6. For those that are needed and not supported, what does it take to get them supported? By whom, in what timeframes?
7. Does the platform and application support variability of user journeys? Can it be driven through data driven templates? Is there clear separation between the user experience, business logic, and the data model ?
8. To what extent are third party developers using the platform to build applications? Does the vendor’s engineering team use the same platform to build its applications? What areas can a SI really change, what can’t they?
9. How does the platform support real time integration to third party systems? Can it assimilate external services and data seamlessly into its processes? Show how these are modeled.
10. Does the application and platform support upgradeability for the future?How will your customizations be preserved, upgraded?
11. What tools do they provide to help the enterprise create and maintain its sales offerings, and manage its relationships to downstream systems? Show how they model your product lifecycle(s).
12. How does the vendor estimate, define and keep score of customer success? How will they support large customers? How is learning built into the process?
13. Does the vendor’s business model, operating model and your contract allow them to take accountability for your success?
14. How do we measure the value created for the enterprise? How willing is the vendor to align the value captured by them with value realized by the enterprise?

It is important for both business and IT teams to understand that irrespective of how compelling the demo is, or how persuasive the vendor may be, applications that are closed, non-extensible and inflexible are impotent, for they cannot be deployed, outside of the most cookie-cutter scenarios. Most enterprises scoring vendors on POCs get this aspect wrong.

Mission-critical, revenue-generating, transactional systems that capture and process orders across channels are the life-blood of the enterprise. To ensure that these systems are world-class and provide maximum differentiation in the marketplace, enterprises need to develop clarity on what they are solving for, what success means, how it translates to participating vendors, and take full ownership.

Just as Mike Allen at Thurman Manufacturing came to painfully realize, the stakes are way too high not to get this right.