How to talk to each other
My very first real job was at HP Research Labs, Palo Alto. Research labs in the late-90s were not really real jobs, though. They were ivory towers where you got to ponder about pie-in-the-sky futuristic problems, build prototypes, write papers and file patents.
But once in a while, an idea would come out of HP-Labs that would change the world. Inkjet and Laserjet printing had been invented in those very hallways, and 3-D printing, nanotech and multi-core processors were being invented while I was there.
My very first project at HP Labs was called E-speak.
E-speak’s problem statement was — how do we make two random systems that don’t know each other beforehand talk to each other ? No pre-known APIs or integration allowed.
Its like two humans who have never met each other before. How do we manage to start a conversation with strangers and do business with them ?
Clearly, you don’t call Accenture for 6-month integration project to buy a cauliflower from the roadside grocer.
If one could build intelligent systems that could figure out each other’s language (protocols) and exchange information appropriately, then a lot of the tedious work that is needed to connect applications together would be unnecessary. We wouldn’t need an Infosys, so that would solve all the unseemly board-room fights, and everyone would be happy.
A superstar team was put together. A lot of ground-breaking progress was made in understanding the problem, papers were published, patents filed and prototypes built. Microsoft saw the work HP was doing and started a similar team called BizTalk.
This was at a time, when the Internet itself was fairly new. Java had just been created. REST and the concept of network APIs did not exist. We first had to figure out how to get things to talk to each other, even assuming everyone knew exactly what they had to talk about.
Service-oriented architecture (SoA) and many of the early standards (SOAP, UDDI etc.) came out of the work that HP and Microsoft did. I represented HP on some of those standards and co-authored a few of them. They look so dated and ugly, that I would never own up to them now — I learnt about craftsmanship later in life.
All the SoA standards did, was solve the easy part of the problem — how to make distributed (but familiar) services find each other and talk to each other across a network. But a solution to the overall stranger problem remained elusive.
HP went through a lot of business problems and HP Labs shrunk. We learnt how tough the problem (even its subsets) was and gave up. The team disbanded and the people moved on to other things. In retrospect, E-speak was way too early. We needed more shoulders to stand on.
With a couple of decades under the bridge, the problem remains unsolved.
Ideas never die.
Yet, the problem remains tantalizing — theoretically possible, practically impossible. Those are the most sadistic ones.
With the rise of bots, the problem becomes increasingly pertinent again. Bots today are mostly feeble B2C attempts where we are trying to make systems talk to humans. Humans are very forgiving in language, but emotionally unforgiving.
But the system-to-system conversation problem is very different. Systems are very unforgiving in language but emotionally very forgiving. They will happily try a thousand different pickup lines to get the other system to swipe right. To truly get a Skynet going — and we do want one, dont we ? — we really need to solve the system-to-system talking problem.
I don’t have a solution, but I have some thoughts on how I would re-attempt E-speak today. But, first a diversion into business systems design which comes in the next post.