Jedi Lessons in Analytics — Segmentation
Episode 2 — Beware the rigidity of thought, you must
You must unlearn what you have learned.
I have a theory about the Force, something that has eluded most of the Jedi (and the fans) for decades.
It might not predict the course of the next film, but it does allow me to discuss some important parallels in analytics while simultaneously talking about Star Wars.
The Force
Let’s begin with Yoda’s description of the Force to Luke on Dagobah, long ago.
For my ally is the Force, and a powerful ally it is. Life creates it, makes it grow. Its energy surrounds us and binds us.
Luminous beings are we, not this crude matter.
You must feel the Force around you, here, between you, me, the tree, the rock, everywhere, yes. Even between the land and the ship.
Yoda says a great deal in a few sentences, and he chooses his words carefully.
Do you see what is not included in this description?
The words light and dark. Or Jedi and Sith.
In the Star Wars mythos, the Force flows through and between everyone and everything. There is no indication that it favors one action over another or renders judgement. It simply is.
Choosing the path
In subsequent conversations, of course, Yoda constantly cautions Luke against the dangers of the Dark Side. He warns Luke of Vader’s turn to the Dark Side. He cautions his young student that “once you start down the dark path, forever will it dominate your destiny. Consume you, it will.”
Yet again, Yoda cautions Luke against succumbing to the darkness. Not about becoming a Sith Lord specifically.
Defining the path
Throughout the movies, we see Sith Lords wield powers that Jedi do not.
Emperor Palpatine was pretty free with his force lightning. Darth Vader would force-choke subordinates at the slightest provocation.
At the same time, we see Jedi Knights regularly enter other beings’ minds without permission. Neither Obi-Wan Kenobi nor Qui-Gon Jin hesitated to impress their will and change people’s minds when they deemed it necessary.
Luke Skywalker’s rescue of his friends in Return of the Jedi hinged on his gaining entry to Jabba’s barge, which he accomplished by manipulating Jabba’s servants.
Invading someone’s mind is not generally considered a ‘good’ thing to do.
Two Orders, Only One Choice
Whatever you think of the The Last Jedi, you have to recognize that Luke was trying to make Rey understand that the legend of the Jedi was doing more harm than good. The Force is what it is. and everyone is connected to it.
The Jedi created their order and set a series of arbitrary rules: suppress emotion, remain celibate, avoid violence when possible, and always protect the innocent. Of those rules, only the latter two really matter in a battle of good vs evil (or light vs dark).
The downside of such a structure is the implication that follows: if Force warriors are divided into Jedi and Sith, then everyone much choose a side.
It’s a bit heavy handed, but this type of division has parallels in analytics.
Segmentation in the Force
These rules created a segmentation in discipline that the Jedi enforced almost ruthlessly. They might try to save someone who was falling towards the Dark Side, but only if that person was willing to embrace their ways fully.
Circling back to our own galaxy, I have seen multiple analytic projects that fall into similar traps. Project teams, both analysts and domain experts, walk into the data discovery phase with a predetermined framework for segmentation of their data. And they consider these predefined boundaries to be absolute.
Bad idea, this is.
Most worthwhile analytic processes are going to involve various areas and axes of segmentation. At its core, segmentation is simply the grouping of data by common factors (e.g., geographic region, annual revenue, seasonal activity). These groupings are extremely useful and should not be ignored.
If your group walks into the discovery process with predefined segments, however, then you will end up twisting or pruning your data to meet those preconceptions. Data should always drive your analysis, not follow it.
The key to successful data discovery is starting with a decent foundation of domain knowledge but absolutely no idea what the data will show.
In Summary
There is a light side to the Force. There is a dark side to the Force.
We know this in Star Wars because Force-sensitive beings have felt it. They sense the pull of each and the consequences of choosing one over the other.
Jedi and Sith, however, are constructs of living beings . They only mean something to the people that follow them.
Additionally, be careful of applying your segmentation too rigidly. When you encounter something that fits none of your defined categories, stop and think. Maybe you need a new category.
Advice for Masters and Apprentices
Walk into you projects with a working knowledge of the Force (or your own business domain as applicable).
Don’t worry so much about how others have defined its leanings.
Keep an open mind and let the data guide you.
For more Jedi Lessons in Analytics…
