There are many nifty libraries in unix / linux. And that is the beauty of the unix / linux system. People have said it is good to look at the different unix functionalities as libraries akin to the libraries in the programming language; think static libraries of C or libraries of Java et al. It is good to think of shell scripting as Programming Interface to Unix though a limited one (more on the same may be in a later post).
Hence one of the feature of all modern Unix systems is service supervision, supervision trees.
s6 is one such tool we use in our systems. Where we use it is to supervise our long running application processes under root through it. The application process actually areJava 7 / 8 process running in the alpine linux.
And we wanted to move to Java 10. Now Java 10 and rather it’s upcoming better LTS support enabled Java 11 has lots of stuff over Java 8, and it’s a leap in its own right from Java 8. (Java 8 biggest leap was Stream and Lambda). Java 10 / 11 gives Module / JPMS / Project Jigsaw, Size and Memory improvements, better and continuously improving GCs, better APIs, better Container / Docker support, new / better concurrency, the list is endless. cutting long story short, we wanted to move to Java 10 for the new mongo driver and its features (more can be asked from dev team), what I know of is the codec and bson / Database DTO / POJO features of new mongo driver available in Java 10 compatible mongo client driver version only (~ 3.5.x )
Now we ran into couple of problems while moving to Java 10.
- Alpine has no support of Java 9 and Java 10. At least not straightforward. Heard that they would be back on track on Java 11.
- Newrelic, the APM we use has no support of Java 11 :) and has support for Java 10. You can see the two opposite forces of nature here, mucj like every thing in nature :) Centripetal and Centrifugal force. :)
- Also ++, s6 the way we have configured or got to work so far was with alpine linux only. That is more like our problem, and not a deficiency or support wanting / in process in the third party OSS we are using. And so we would go in that direction and fix out this issue.
In the end we decided to take an official openjdk:jdk-10 image and go with the newrelic support, ditched alpine, and for the time being moved away from s6 (again would relook asap, preferably in couple of days time) and at least go in prod.
Now, in this endeavour we came to know more about s6 and the history, and architecture of it. I would recommend everyone to go and read its doc. Its kind of technical documentation that is rare to find these days, and explain the rationale, high level concepts, library documentation, and alternatives quite lucidly. I wouldn’t ask anyone (or rather documentation also doesn’t I think) to just make it as the supervisor process for each and every use case and demean others, but its still a good comprehensive study.
The salient points of s6 are:
- It single responsibility / single problem solving approach to service supervision.
- It doesn’t mix it with other concerns like machine management etc.
- It makes the supervision as process 1 and argues that that actually should be the way to go for service supervision.
- One service to One daemon rule.
- It uses a decoupled architecture where service communication never happens directly but through s6.
- Among other things also round off why .pid or the other side of the coin; non-supervision system lack / not going to be the solution.
On the conclusion, s6 does seem a bit opinionated (or may be for beginner unix user like me :)) but in a good way. And we have to solve the issue of ours.
Would go in further details in s6 in future and where else we can use it, the home page itself mentions syslogd, or system V init is quite legacy etc.
Till then happy reading, enjoy!
