Coming back to the UNIX Philosophy, again and again
For I am convinced that the UNIX Philosophy will be the successor of IT
Since the age of 12 I am into computer science, and I can admit since then I’ve learned a bunch of things. I do love innovating and creating new things that’s actually one reason why I also love to do startup-stuff and to evaluate business processes and technologies. But that’s another story.
While developing a framework for large-scale log-data storage, processing and analysis with different cloud storage and computing systems, and the comparison of Amazon Elastic MapReduce and S3 versus Joyent Manta — two completely different internet facing object stores with computing power, I (again) stumbled upon the very famous Jon Bentley’s quest:
Read a file of text, determine the n most frequently used words, and print out a sorted list of those words along with their frequencies.
(Google had nearly the same question, with the URL frequency count problem posed in section 2.3 of Google’s MapReduce paper. )
Donald Knuth developed a monolithic system (10+ pages of Pascal), solving the problem very intelligibly but Doug McIllroy once wrote:
Knuth has shown us here how to program intelligibly, but not wisely. I buy the discipline. I do not buy the result. He has fashioned a sort of industrial-strength Fabergé egg — intricate, wonderfully worked, refined beyond all ordinary desires, a museum piece from the start.
Of cause McIllroy had to come up with a simpler solution, and he nailed it:
This is a complete replacement of Knuth’s solution. This nifty six liner uses three different tools, each one does exactly one thing and it does it very well (that’s the UNIX Philosophy): tr, sort, uniq, and sed. But what McIllroy criticized was the monolithic system:
Now, in 1968, I would have thought he was doing just right. He was taking this sub-routine and that sub-routine, and putting them together in one program. Now, I don‘t think that is just right. I think that the right way to do that job is as we do it in Unix, in several programs, in several stages, keeping their identity separate, except in cases where efficiency is of extreme importance. You never put the parts into more intimate contact. It’s silly. Because, once you’ve got them there, it’s hard to get them apart. You want to change from English to Norwegian, you have to go way to the heart of Knuth’s program. You really ought to be able to just change the pre-processors that recognize this is a different alphabet.
And that’s what I experience often, people try to solve hard problems through hard work. But most of the time it is more wisely to step back once and think about the solution.
The problem of monolithic systems and cloud computing has been researched by Martin Fowler as well:
Monolithic applications can be successful, but increasingly people are feeling frustrations with them — especially as more applications are being deployed to the cloud . Change cycles are tied together — a change made to a small part of the application, requires the entire monolith to be rebuilt and deployed.
[…]
These frustrations have led to the microservice architectural style: building applications as suites of services. As well as the fact that services are independently deployable and scalable, each service also provides a firm module boundary, even allowing for different services to be written in different programming languages. They can also be managed by different teams .
Ok, what Martin Fowler points out sounds a bit like the UNIX Philosophy:
Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.
The mindset of “microservices” is one reason why I am convinced that the UNIX Philosophy will be the successor of IT.
Bottom line, search for smart solutions. Step back and think.
Thanks to @rradczewski for reading through the draft!
