Shadow IT: An Organic Response to Lowest-Common Denominator Management

The rise of Shadow IT shouldn’t really be a surprise to anyone given the dominant management paradigms in place in most IT shops across the world. The truth is that most of our IT practices revolve around a lowest-common-denominator approach to application support and deployment. But the absence of catastrophe is not the same as a positive user experience. And so users are looking outside their IT cost centers for other ways to meet their requirements.

When a new application is requested, the IT mechanics all revolve around requirements. What is the minimum compute capacity required? What kind of server and OS are needed? How much storage is required? How many users will access the applications? How many 10GE ports need to be connected to the network? If all of these minimum requirements are satisfied, we can declare success.

After that, the best anyone can do is just avoid calamity. This is why most IT metrics revolve around things like capacity (can you service everything?) and availability (is it always up?).

But end user experience is not determined by minimum bars. If it was, our collective view of the airline industry would be dramatically different. Consider that we are in what is statistically the safest era of air travel ever, and yet dissatisfaction with airlines is at an all-time high. As end users, we don’t measure success by the absence of failure. We demand more.

So how do you navigate this type of environment?

Managing satisfaction is an expectation-matching game. One of the core issues in IT is that there is no explicit contract between IT and the end users it serves. And the implicit agreement that exists differs depending on which side you are on.

IT has to support any and all applications. For large enterprises, this can number in the thousands. The applications are owned by various lines of business, but they all must function. This puts an enormous tax on the underlying infrastructure, and forces IT to move at a glacial pace. The kinds of changes that would drive meaningful improvement don’t necessarily lend themselves well to backward compatibility. It is simply unreasonable to think that you can move forever forward (and at an accelerated rate!) while carrying all the legacy baggage of the past two decades. In this environment, it is a herculean task just to keep everything running. The fact that IT can meet uptime requirements is deserving of praise!

On the user side, satisfaction is dominated by negative experience. You might drive your car 100,000 miles without issue, but you will remember with remarkable clarity the time you were stranded in the middle of nowhere because of a timing belt failure. That one experience could be enough to steer you another direction when you buy your next car. but negative experience is not limited to catastrophic failure. If your airline food is bad, if your hotel wireless doesn’t work, or if your application is unresponsive when you are crunching on a deadline… these all shape your experience.

The dynamic we have created is one of mismatched expectations. And without anything that is explicit and well-understood, we have little hope of getting the transparency required to improve things.

What if instead of a lowest-common-denominator approach that promotes uptime at the expense of everything else, there were specific SLAs for the most critical applications? It might be that some applications like trading or data processing are particularly sensitive to response time for certain workloads. It could be that others like voice and video are sensitive to loss and jitter. It might be that some things like batch replication jobs are really about consistency. If these SLAs were understood, agreed upon, and documented, we would at least have a framework for evaluating success beyond mere availability.

There are several challenges with this approach though. First, experience must be expressed in terms the user understands. A user is unlikely to set some jitter SLA. This means that IT must translate these requirements into terms that are meaningful to the underlying infrastructure. Second, experience is an overarching thing, which doesn’t lend itself particularly well to a silo’d IT environment. You need to measure SLAs inclusive of compute, storage, networking, and applications. And finally, managing these SLAs requires a different level of instrumentation than currently exists in most environments today. Orchestrating data collection and correlation, and then using that to make meaningful sense of performance is non-trivial.

But despite being difficult, this type of measurement ought not be impossible. From where most people are today, this might seem like a distant fantasy. But if IT continues to measure itself against a lowest common denominator, it will find that it has moved from an active participant in driving value to a pure governance and auditing function of Shadow IT initiatives. I doubt this outcome is good for either side.

[Today’s fun fact: The most money ever paid for a cow in an auction was $1.3 million. That must be one sweet-tasting burger.]