Collaboration… Inception style
How NYC unexpectedly redefined customer-centric delivery
Here is the setup: you want to take Agile collaboration to the next level and dramatically shorten delivery sprints with your customer
But who really is the customer? Everyone says they want to be customer-centric but in order to do that you must be flexible enough to meet their needs. Fortunately, software is malleable enough to do that — but the catch is using the right technology to structure things accordingly. In fact, I would suggest your development team is an extension of your customer and I will attempt to illustrate this.
So far in this series, I have compared CodeBuilder technology to the movie Inception and ‘entering’ the mind of the machine.
Let’s now focus on entering the mind of the ‘customer’.
The Gemba Walk
When I worked in Japan, I learned firsthand about Gemba — a LEAN concept that translates to “visiting the crime scene”. It informally means going where sh** gets real. Toyota broke through cultural barriers by having managers leave their offices and walk the factory floor to better understand the real-world battles their workers were up against. That is, the Product Owner was willing to get dirty and work hands-on with the team. In this sense, the people on the factory floor were the first “customers” of the Toyota build process.
Mobile: Software’s Crime Scene?
I love global travel, so after Japan my Gemba walk took me about as far away from the West Coast and NYC as possible — to places like subarctic Canada, dangerous parts of Africa and deepwater offshore. ‘Edge’ / IoT software development is extreme in about every way possible — mobile devices talk to servers that are themselves mobile. Long workdays in NYC/San Francisco pale in comparison to the “hitch” schedule: developers and end customers are together almost 24x7.
This is what extreme Agile collaboration looks like: customers (the paying kind) and developers leave well before ‘dawn’ with temperatures far below zero, ride together several hours on icy mountain roads to a work trailer, work all day (in muddy boots) with various users and stakeholders, drive back together, eat dinner together and then work together around the kitchen table until 2AM — and then back up at 6AM to do it all over again. If space is tight, customers and devs might bunk in the same room — no breaks, no weekends. You can’t get much more customer-focused than this. Despite flaky internet, communication flows freely so sprints happen in hours not days or weeks. Only an informal taskboard is necessary because there is zero question about what the customer needs. However, it is not long before the ‘edge’ developers in the field are finding themselves trying to explain detailed technical requirements to folks back home — or find it faster to simply code it themselves.
Redefining Customer-Centric Agile
When your office is a metal cage in Africa, saying ‘no’ to the (real) customer across a tiny desk doesn’t work—you aren’t exactly leaving the room and if you get up for more chicory coffee, you better sit down with an answer. After all, your software is responsible for the safety and operation of equipment somewhere 12,500 feet beneath your laptop, spread over 500 miles of tunnels.
Even if you have phone service, there is no privacy to make a call (hence our teams were using online chat years before Slack was a thing). You are either ‘collaborating’ or sneaking off to the restroom… which has a hole in the ground where the toilet should be. As you stare into that hole, you see everything the customer sees and start to question basic tenets of the software development process.
This is going where sh** gets real.
In such conditions, the hopelessness of trying to build software via traditional methods becomes obvious. Blaming the customer is about as productive as cursing the stormy sea for heaving around a billion dollar floating rig. Or (in the case of Wall Street) a global market crash.
Software technology must be able to adapt.
Agile breaks down when most of the team is relaxing in trendy ‘collaborative’ workspaces while the PO is fighting malaria and armed road bandits. Don’t get me wrong — I enjoy Virgin Upper Class, porcini-rubbed Delmonico and 25-year Dalmore as much as anyone — but our software culture has gotten closer to something almost the opposite of Gemba and IMO a big unspoken reason why so many Agile projects struggle. As software professionals, it might be our turn to leave our padded offices and visit the ‘factory floor’.
Adversity: Genesis of Innovation
Of course, I’m not just talking about collaboration with humans. But like Toyota discovered, working in these extreme conditions caused me to research better ways to building software and ultimately arrive at NYC’s rethinking of enterprise Agile. I saw right away how the software workflow is littered with labor-intensive inefficiencies. Developers must work through outdated file-based methodologies and deal with jarring edit-run processes (with only limited hot loading capabilities). While that may have been acceptable decades ago, Singh and Higgins proved there are better alternatives.
In short, the CodeBuilder ‘unbundling’ layer discussed previously also acts as a collaboration platform / workbench:
What sort of collaboration is possible? Users sitting in a meeting can see changes immediately and provide feedback, which is critical in remote locations. Testers can immediately see code changes and start writing test cases. It also means AI automation can see your code at point of use and become part of the natural workflow (Human-In-The-Loop). Integration is nearly instantaneous. Software teams in offices around the world can build together in realtime without trying to share desktops or editor screens.
Behind the scenes, a CodeBuilder is essentially a matrix of distributed code clusters with hot loading/replication. Just like a database, developers start with an ‘empty’ but live system. Development, testing, user feedback, integration, DevOps is all within the same fabric along with various databases/data sources. Lots of impedance mismatch is eliminated. Moreover, the concept of traditional isolated “debugging” goes away because people can simply log into the matrix to all see what is going on and fix things.
Collaboration Gone Wild
Athena and Quartz are considered the largest Python codebases in the world (20M+ lines) with 5K active developers. That is comparable to Facebook’s massive Mercurial CI system, although both were built years earlier and with arguably far greater complexity.
While SecDB’s claim to fame was overcoming serious technical hurdles, Athena is most known for the challenges of rolling out scaled Agile across a massive enterprise years before there was such a term.
Of course both SecDB and Athena are aging by today’s standards — and everyone has their own idea of what a “code builder” looks like— but NYC is currently locked in an arms race to take this disruptive technology to the next level.
Part 5 is here.