Hitchhiker’s Guide to Analytics — Improbability Drive
2 to the power of 800,000 to 1 against, and falling…
The Hitchhiker’s Guide to the Galaxy series showed a lot of things about how Douglas Adams viewed the world. He saw the pieces, but I’m convinced that he wasn’t entirely pleased with quite how they were put together.
So he reorganized them, just a bit.
This is the “other side” of analytics: the process of synthesis. Once you have deconstructed your subject matter into its component pieces, you reassemble them in the manner that provides the most value.
Most of us are trying to provide insight into reality. Adams had a different goal but a similar method.
Douglas Adams needed a new and clever way to travel through space. Other writers had given us warp drives, hyperspace, and a flood of other methods for faster-than-light travel with one common theme: they were presented as possible extensions of the laws of space and time as we know them.
Adams told the laws of space and time to go hang. He had work to do.
Heart of Gold
2 to the power of 600,000 to one against, and falling…
Please ignore the second sun in the sky. It will probably be gone before you get burned.
If you’ve heard of the Guide, then you’ve heard about the Heart of Gold. It was the first ship in the Galaxy to use the new Infinite Improbability Drive, which allowed the ship to pass through every conceivable point in the universe without all that tedious mucking about in hyperspace.
Our heroes did not zip through colorful portals or stretch into infinity. The Heart of Gold simply appeared at the given coordinates (again, after passing through every conceivable point in the universe) simply by calculating exactly how improbable it was that it would be there.
Think about the brilliance of that connection. The ship didn’t have to calculate a path to its destination; it just needed to determine precisely how unlikely it was to be able to do so.
Along the way, the crew (and the universe) were inevitably subjected to any number of odd coincidences, most of which were virtually impossible.
Invention
2 to the power of 400,000 to 1 against, and falling…
The creation of the Infinite Improbability Drive was equally clever. I quote:
The principle of generating small amounts of finite improbability by simply hooking the logic circuits of a Bambleweeny 57 sub-meson Brain to an atomic vector plotter suspended in a strong Brownian Motion producer (say a nice hot cup of tea) were of course well understood.
I’ll summarize the next bit, shall I?
Most respectable physicists were frustrated, partially because this was a misuse of science, but mostly because they didn’t get invited to those kind of parties. Also, they couldn’t successfully generate the infinite improbability needed to fling a ship across the vast distances between stars. Finally, they declared it a virtual impossibility and went home.
One night, a student left to clean up after a particularly unsuccessful party, found himself reasoning as follows: if such a machine is a virtual impossibility, it must be a finite improbability. So all I’d need to do is work out exactly how improbable it is, feed that number into the finite improbability generator, give it a fresh cup of really hot tea, and turn it on!
He was then rather startled to discover that he’d created the long-sought Infinite Improbability Drive out of thin air.
Back to quoting:
It startled him even more when, just after he was awarded the Galactic Institute’s Prize for Extreme Cleverness, he got lynched by a rampaging mob of respectable physicists who had finally realized that the one thing they really couldn’t stand was a smart-ass.
That, my friends, is an excellent example of research successfully applied.
Back to the story
2 to the power of 200,000 against, and falling…
We’ve passed the threshold at which an individual, stranded in space with no spacesuit or air supply, could expect to be saved by a passing ship.
That number in the Guide is 2 to the power of 267,709 against, based on the universe in which it occurred. It’s also the number of an Islington flat at which Arthur Dent had recently attended a very good party, at which he had some very good drinks and met a very nice girl he totally failed to impress.
After being thrown from a Vogon airlock, Arthur Dent and Ford Prefect are picked up by the Heart of Gold, piloted by Zaphod Breeblebrox and Trillian.
Trillian was the aforementioned very nice girl. the only other Earthling to survive the destruction of the Earth by the Vogons (spoiler!).
She met Zaphod at the same party (he was traveling incognito).
Zaphod was, by another meaningless coincidence, Ford’s semi-cousin.
One of the side effects of the Infinite Improbability Drive was a constant stream of meaningless coincidences.
Moving on
2 to the power of 100,000 to 1 against, and falling…
At this point, you might find yourself turning into a penguin. Stop it.
Douglas Adams created a spaceship powered by Improbability, and he paid enough attention to the details to make it believable. If everyone in analytics (Analysts? Analyticists? Data scientists? Analytic engineers?) paid as much attention on the follow through, they would create better solutions.
Ha! Weren’t expecting me to pull it back that quickly, were you?
Really, though, no wonder we have trouble with job titles.
Applied Analytics
2 to the power of 20,000 to 1 against, and falling…
This might be the probability that you’re still paying attention. Mostly, I’m just hoping I don’t call any unfortunate yet fatalistic petunias into existence.
The thing is, I’ve been an analyst by trade but not by nature. I’m usually more focused on the other side of the analytic equation (Synthesyst? Syntheliticyst? You get the idea).
I am interested in understanding what’s in front of me, but I’m more interested in what I can do with it. And how to do it properly.
I’m an architect. I’m not sure it is possible to be a good data analyst without also being something of an architect, but I am sure that it is highly improbable for anyone to be a good architect without also being a skilled analyst. One needs to see the pieces clearly in order to make use of them.
It’s not a competition; it’s a matter of interest and inclination.
I’m not sure just how improbable. Even if I were, I don’t have the Heart of Gold to make use of that information.
That’s why I see the Infinite Improbability Drive not only as a clever device to power spaceships (and plot lines), but also as a fascinating application of probability. Physicists are constantly looking for new dimensions. Douglas Adams took something we already had and declared it one.
2 to the power of 1000 against…
2 to the power of 100 against…
10,000 to 1 against…
10 to 1 against…
4 to 1 against…
2 to 1 against…
Probability factor of 1 to 1 achieved. We have normality.
Anything you still cannot deal with is therefore your own problem.
