How to Communicate with Containerd Using Java

Credit containerd.io

In the last post I described how to setup containerd.

containerd is used as plumbing layer of other container management platforms such as Docke rand Kubernetes, but it does not mean we can not interact with it directly. In fact it comes with its own command line client ctr which we used at the end of last post.

In this exercise, I’d like to go a bit deeper and explore how to connect to containerd and run operations on itusing the GRPC interface with Java.

Start a Maven project :

mvn archetype:generate -DgroupId=io.mark.containerd.client.ex1 -DartifactId=java-containerd-client-ex1 -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false

Make modifications to pom.xml

1. Added the 3 dependencies to <dependencies> section

<dependency>
 <groupId>io.grpc</groupId>
 <artifactId>grpc-netty</artifactId>
 <version>1.21.0</version>
 </dependency>
 <dependency>
 <groupId>io.grpc</groupId>
 <artifactId>grpc-protobuf</artifactId>
 <version>1.21.0</version>
 </dependency>
 <dependency>
 <groupId>io.grpc</groupId>
 <artifactId>grpc-stub</artifactId>
 <version>1.21.0</version>
 </dependency>
 <dependency>

2. Add a build section as below:

<build>
 <extensions>
 <extension>
 <groupId>kr.motd.maven</groupId>
 <artifactId>os-maven-plugin</artifactId>
 <version>1.5.0.Final</version>
 </extension>
 </extensions>
 <plugins>
 <plugin>
 <groupId>org.xolstice.maven.plugins</groupId>
 <artifactId>protobuf-maven-plugin</artifactId>
 <version>0.5.1</version>
 <configuration>
 <protocArtifact>com.google.protobuf:protoc:3.7.1:exe:${os.detected.classifier}</protocArtifact>
 <pluginId>grpc-java</pluginId>
 <pluginArtifact>io.grpc:protoc-gen-grpc-java:1.21.0:exe:${os.detected.classifier}</pluginArtifact>
 </configuration>
 <executions>
 <execution>
 <goals>
 <goal>compile</goal>
 <goal>compile-custom</goal>
 </goals>
 </execution>
 </executions>
 </plugin>
 </plugins>
 </build>

Run package to make sure it builds. It downloads the dependencies and runs the package.

mvn clean package
[INFO] Building jar: /home/centos/Java/examples/java-containerd-client-ex1/target/java-containerd-client-ex1–1.0-SNAPSHOT.jar
[INFO] — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 
[INFO] BUILD SUCCESS
[INFO] — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 
[INFO] Total time: 7.399 s

We need to clone 2 repos — containerd repo and gogoproto . Containerd repo contains the proto service definition files (such as image service) and gogoproto is imported by those, so we need those as well

mkdir -p /tmp/containerd_protos/github.com/containerd
cd /tmp/containerd_protos/github.com/containerd
git clone https://github.com/containerd/containerd.git
cd /tmp/containerd_protos/
git clone https://github.com/gogo/protobuf.git

At his point we need to configure the protobuf-maven-plugin. First we need to change the protoSourceRoot to point to the base where we cloned repos.

<protoSourceRoot>/tmp/containerd_protos/</protoSourceRoot>

We’ll work on the images API for this exercise so let’s add includes into configuration section of protobuf-maven-plugin for the directory where images.proto exists. We also need types proto files which are used by the images (and other services) uses, plus the gogoproto definitions. configuration section looks like this after all these changes

<configuration>
<protocArtifact>com.google.protobuf:protoc:3.7.1:exe:${os.detected.classifier}</protocArtifact>
 <pluginId>grpc-java</pluginId>
 <pluginArtifact>io.grpc:protoc-gen-grpc-java:1.21.0:exe:${os.detected.classifier}</pluginArtifact>
 <protoSourceRoot>/tmp/containerd_protos/</protoSourceRoot>
 <includes>
<include>github.com/containerd/containerd/api/services/images/v1/*.proto</include>
<include>github.com/containerd/containerd/api/types/*.proto</include>
 <include>gogoproto/*.proto</include>
 </includes>
 </configuration>

At this point, when we run mvn package, it should run successfully and generate the java classes from the proto files

$ mvn -DskipTests package
[INFO] Scanning for projects…
[INFO] — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 
[INFO] Detecting the operating system and CPU architecture
[INFO] — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 
[INFO] os.detected.name: linux
[INFO] os.detected.arch: x86_64
[INFO] os.detected.version: 3.10
[INFO] os.detected.version.major: 3
[INFO] os.detected.version.minor: 10
[INFO] os.detected.release: centos
[INFO] os.detected.release.version: 7
[INFO] os.detected.release.like.centos: true
[INFO] os.detected.release.like.rhel: true
[INFO] os.detected.release.like.fedora: true
[INFO] os.detected.classifier: linux-x86_64
[INFO]
[INFO] — — — < io.mark.containerd.client.ex1:java-containerd-client-ex1 > — — — 
[INFO] Building java-containerd-client-ex1 1.0-SNAPSHOT
[INFO] — — — — — — — — — — — — — — — — [ jar ] — — — — — — — — — — — — — — — — -
[INFO]
[INFO] — — protobuf-maven-plugin:0.5.1:compile (default) @ java-containerd-client-ex1 — -
[INFO] Compiling 6 proto file(s) to /home/centos/Java/examples/java-containerd-client-ex1/target/generated-sources/protobuf/java
[INFO]
[INFO] — — protobuf-maven-plugin:0.5.1:compile-custom (default) @ java-containerd-client-ex1 — -
[INFO] Compiling 6 proto file(s) to /home/centos/Java/examples/java-containerd-client-ex1/target/generated-sources/protobuf/grpc-java
[INFO]
[INFO] — — maven-resources-plugin:2.6:resources (default-resources) @ java-containerd-client-ex1 — -
[INFO] Using ‘UTF-8’ encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/centos/Java/examples/java-containerd-client-ex1/src/main/resources
[INFO] Copying 6 resources
[INFO] Copying 6 resources
[INFO]
[INFO] — — maven-compiler-plugin:3.1:compile (default-compile) @ java-containerd-client-ex1 — -
[INFO] Changes detected — recompiling the module!
[INFO] Compiling 8 source files to /home/centos/Java/examples/java-containerd-client-ex1/target/classes
[INFO]
[INFO] — — maven-resources-plugin:2.6:testResources (default-testResources) @ java-containerd-client-ex1 — -
[INFO] Using ‘UTF-8’ encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/centos/Java/examples/java-containerd-client-ex1/src/test/resources
[INFO]
[INFO] — — maven-compiler-plugin:3.1:testCompile (default-testCompile) @ java-containerd-client-ex1 — -
[INFO] Nothing to compile — all classes are up to date
[INFO]
[INFO] — — maven-surefire-plugin:2.12.4:test (default-test) @ java-containerd-client-ex1 — -
[INFO] Tests are skipped.
[INFO]
[INFO] — — maven-jar-plugin:2.4:jar (default-jar) @ java-containerd-client-ex1 — -
[INFO] Building jar: /home/centos/Java/examples/java-containerd-client-ex1/target/java-containerd-client-ex1–1.0-SNAPSHOT.jar
[INFO] — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 
[INFO] BUILD SUCCESS
[INFO] — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 
[INFO] Total time: 4.735 s

In our project directory, when we go into ‘target’, we see the directories and files generated.

$cd target/
$ ls
classes generated-test-sources maven-archiver protoc-dependencies test-classes
generated-sources java-containerd-client-ex1–1.0-SNAPSHOT.jar maven-status protoc-plugins

Under generated-sources, we can find a few files including the ImagesOuterClass.java which contains our protobuf helper code and ImagesGrpc.java which contain the grpc helper code

Before we start creating our client, we need to add one more dependency: native platform specific JNI transport from Netty:

<dependency>
 <groupId>io.netty</groupId>
 <artifactId>netty-transport-native-epoll</artifactId>
 <version>4.1.36.Final</version>
 <classifier>${os.detected.classifier}</classifier>
 </dependency>

This is needed if we would like to use the UNIX socket since Java does not natively support Unix sockets. As we configured that in our containerd installation (default setup), we need the Netty native transport here.

$ cat /etc/containerd/config.toml
root = “/var/lib/containerd”
state = “/run/containerd”
……
[grpc]
 address = “/run/containerd/containerd.sock”
……

I also user container command line client ctr to pull some images, so we can query them from our java client. This is what the environment looks like :

#/usr/local/bin/ctr namespaces ls    (List the namespaces we have)
NAME LABELS
default
examplectr
# /usr/local/bin/ctr -namespace examplectr images ls (list the images in our examplectr namespace)
REF TYPE DIGEST SIZE PLATFORMS LABELS
docker.io/library/alpine:3.6 application/vnd.docker.distribution.manifest.list.v2+json sha256:66790a2b79e1ea3e1dabac43990c54aca5d1ddf268d9a5a0285e4167c8b24475 1.9 MiB linux/386,linux/amd64,linux/arm/v6,linux/arm64/v8,linux/ppc64le,linux/s390x -
docker.io/library/alpine:latest application/vnd.docker.distribution.manifest.list.v2+json sha256:769fddc7cc2f0a1c35abb2f91432e8beecf83916c421420e6a6da9f8975464b6 2.6 MiB linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/ppc64le,linux/s390x -
docker.io/library/httpd:alpine application/vnd.docker.distribution.manifest.list.v2+json sha256:8a3c608e3e87ab4b818374a5414e63b63097c248693b402dac3616f2ce6d487b 39.6 MiB linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/ppc64le,linux/s390x -

As can be seen above, I have already 3 images in the ‘examplectr’ namespace

Now, time to start coding our client code

We’d like to list the images within the namespace. The definition of this Part of images.proto which defines the Images service, List rpc call and associated Request and Response objects is below:

package containerd.services.images.v1;
service Images {
// List returns a list of all images known to containerd.
 rpc List(ListImagesRequest) returns (ListImagesResponse);
}
message Image {
 // Name provides a unique name for the image.
 //
 // Containerd treats this as the primary identifier.
 string name = 1;
// Labels provides free form labels for the image. These are runtime only
 map<string, string> labels = 2;
// Target describes the content entry point of the image.
 containerd.types.Descriptor target = 3 [(gogoproto.nullable) = false];
// CreatedAt is the time the image was first created.
 google.protobuf.Timestamp created_at = 7 [(gogoproto.stdtime) = true, (gogoproto.nullable) = false];
// UpdatedAt is the last time the image was mutated.
 google.protobuf.Timestamp updated_at = 8 [(gogoproto.stdtime) = true, (gogoproto.nullable) = false];
}
message ListImagesRequest {
 // Filters contains one or more filters using the syntax defined in the containerd filter package.
 // If filters is zero-length or nil, all items will be returned.
 repeated string filters = 1;
}
message ListImagesResponse {
 repeated Image images = 1 [(gogoproto.nullable) = false];
}

I created a file src/main/java/io/mark/containerd/client/ex1/ContainerdClientExample.java with content:

package io.mark.containerd.client;
import containerd.services.images.v1.*;
import containerd.services.images.v1.*;
import io.grpc.*;
import io.grpc.stub.*;
import io.netty.channel.epoll.EpollEventLoopGroup;
import io.netty.channel.EventLoopGroup;
import io.grpc.netty.*;
import io.grpc.Context;
import io.grpc.stub.MetadataUtils;
import io.grpc.Metadata;
import java.util.List;
import containerd.services.images.v1.ImagesOuterClass.Image;
public class ContainerdClientExample
{
 public static void main( String[] args ) throws Exception
 {
 // Create a new channel using Netty Native transport
 EventLoopGroup elg = new EpollEventLoopGroup();
 ManagedChannel channel = NettyChannelBuilder
 .forAddress(new io.netty.channel.unix.DomainSocketAddress(“/run/containerd/containerd.sock”))
 .eventLoopGroup(elg)
 .channelType(io.netty.channel.epoll.EpollDomainSocketChannel.class)
 .usePlaintext(true)
 .build();
// Since containerd requires a namespace to be specified when making a GRPC call, we will define a header with “containerd-namespace” key, set the value to our namespace
 Metadata header=new Metadata();
 Metadata.Key<String> key =
 Metadata.Key.of(“containerd-namespace”, Metadata.ASCII_STRING_MARSHALLER);
 header.put(key,”examplectr”);
//Create the stub and attach the header created above
 ImagesGrpc.ImagesStub stub = ImagesGrpc.newStub(channel);
 stub = MetadataUtils.attachHeaders(stub, header);
//Let’s build the ListImagesRequest with no filter
 ImagesOuterClass.ListImagesRequest request =
 ImagesOuterClass.ListImagesRequest.newBuilder()
 .addFilters(“”)
 .build();
System.out.println(“==============================================================”);
 System.out.println(“IMAGE\n============”);
 // Make the RPC Call
 stub.list(request, new StreamObserver<ImagesOuterClass.ListImagesResponse>() {
 // When response is received iterate over the Response and print the names of images
 public void onNext(ImagesOuterClass.ListImagesResponse response) {
 List<Image> images= response.getImagesList();
 for (int i=0;i<images.size();i++) {
 System.out.println(i+”-”+images.get(i).getName());
 }
 }
 // if there is an error
 public void onError(Throwable t) {
 t.printStackTrace();
 }
 // when server completes the response and closes the stream shutdown our channel and EventLoopGroup
 public void onCompleted() {
 channel.shutdownNow();
 elg.shutdownGracefully(50,50,java.util.concurrent.TimeUnit.MILLISECONDS);
 }
 });
}
}

And let’s run this:

$ sudo mvn -DskipTests exec:java -Dexec.mainClass=io.mark.containerd.client.ContainerdClientExample (Need to run with sudo as the socket file is owned by root. It can be configured with proper group ownership to execute as centos or any other user)
[INFO] Scanning for projects…
[INFO] — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 
…….
[INFO] — — exec-maven-plugin:1.6.0:java (default-cli) @ java-containerd-client-ex1 — -
==============================================================
IMAGE
============
Jun 10, 2019 8:33:36 PM io.netty.bootstrap.AbstractBootstrap setChannelOption
WARNING: Unknown channel option ‘SO_KEEPALIVE’ for channel ‘[id: 0xd488f6eb]’
0-docker.io/library/alpine:3.6
1-docker.io/library/alpine:latest
2-docker.io/library/httpd:alpine
[INFO] — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 
[INFO] BUILD SUCCESS
[INFO] — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 
[INFO] Total time: 4.066s