In PART1 we setup a SNO (Single Node OpenShift) cluster — the manual and disconnected way.
In PART2 — which is this article, we’ll add in operators to our disconnected OpenShift installation.
We now have a disconnected cluster that was installed using our local mirror-registry, but as mentioned at the end of PART1 — we don’t yet have access to any operators, so our cluster is essentially useless…
Let’s FIX that, but first let’s note that NOT all operators support a disconnected installation — so to check which do — we can reference the operator catalog.
We’ll need:
1.) The SNO cluster we built in PART1 (along with our local mirror-registry to push the operators to)
2.) SPACE — operators are chunky — make sure you have (quite) a bit of free space to store them (we’re talking hundreds of GB’s to TB’s of space depending on what / how many operators you want to mirror).
3.) The OPM client
The Setup:
We’ll grab the OPM client, extract it and move it to our PATH.
# wget https://mirror.openshift.com/pub/openshift-v4/x86_64/clients/ocp/4.10.37/opm-linux-4.10.37.tar.gz
# tar -xvzf opm-linux-4.10.37.tar.gz
# mv opm /usr/local/bin/We’ll disable the default OperatorHub sources (so we no longer try and look to the internet, which we don’t have access to, for operators).
# export KUBECONFIG=/root/Openshift/configs/auth/kubeconfig
# oc patch OperatorHub cluster --type json -p '[{"op": "add", "path": "/spec/disableAllDefaultSources", "value": true}]'We’ll login to both registry.redhat.io and our local mirror-repository — so we can PULL (from the internet) and PUSH (to our local mirror-registry) the required operator images needed.
# podman login registry.redhat.io
Username: ...
Password:
Login Succeeded!
# podman login installer.ocp.jerome.co.za:8443 --tls-verify=false
Username: init
Password:
Login Succeeded!
We’ll need to mirror our content, but first we need to know what operators we want, their names and possibly their versions if we want to be super specific.
We can do this in several ways, but I’ll cover two for now.
The first — via an existing cluster (if you have one). If not, no sweat — the OpenShift learning labs have a demo cluster we can play with — and as such we can use it to pull out our operator names / versions.
* thanks go to Andrew Sullivan and the “Ask an OpenShift Admin” crew for this tip.
I’ll just focus on grabbing the elasticsearch-operator.
We can even go one better here and grab specific versions of an operator! (we’ll ignore versioning for now, but it’s a “good to know”).
# oc get packagemanifests | grep 'Red Hat Operators' | awk '{print $1}' | xargs -L1 oc get packagemanifests -o yaml | grep currentCSV: | awk '{print $3}' | sort | grep -i elastic
elasticsearch-operator.5.3.9
elasticsearch-operator.5.4.8
elasticsearch-operator.5.5.4
elasticsearch-operator.5.5.4The second — via the redhat-operator-index and grpcurl — as an example here.
# podman run -d -p50051:50051 --name redhat-operator-index -it registry.redhat.io/redhat/redhat-operator-index:v4.10
# wget https://github.com/fullstorydev/grpcurl/releases/download/v1.8.7/grpcurl_1.8.7_linux_x86_64.tar.gz
# tar -xvzf grpcurl_1.8.7_linux_x86_64.tar.gz
# mv grpcurl /usr/local/bin
# grpcurl -plaintext localhost:50051 api.Registry/ListPackages > packages.out
# grep -i elastic packages.out
"name": "elasticsearch-operator"
# podman rm -f redhat-operator-indexNow that we have the operator name, we’ll grab the operator index, prune it to our required operator/s…
# opm index prune -f registry.redhat.io/redhat/redhat-operator-index:v4.10 -p elasticsearch-operator -t installer.ocp.jerome.co.za:8443/redhat/redhat-operator-index:v4.10
WARN[0000] DEPRECATION NOTICE:
Sqlite-based catalogs and their related subcommands are deprecated. Support for
them will be removed in a future release. Please migrate your catalog workflows
to the new file-based catalog format.
INFO[0000] pruning the index packages="[elasticsearch-operator]"
INFO[0000] Pulling previous image registry.redhat.io/redhat/redhat-operator-index:v4.10 to get metadata packages="[elasticsearch-operator]"
INFO[0000] running /usr/bin/podman pull registry.redhat.io/redhat/redhat-operator-index:v4.10 packages="[elasticsearch-operator]"
INFO[0008] running /usr/bin/podman pull registry.redhat.io/redhat/redhat-operator-index:v4.10 packages="[elasticsearch-operator]"
INFO[0014] Getting label data from previous image packages="[elasticsearch-operator]"
INFO[0014] running podman inspect packages="[elasticsearch-operator]"
INFO[0014] running podman create packages="[elasticsearch-operator]"
INFO[0015] running podman cp packages="[elasticsearch-operator]"
INFO[0032] running podman rm packages="[elasticsearch-operator]"
INFO[0033] deleting packages pkg=3scale-operator
INFO[0033] packages: [3scale-operator] pkg=3scale-operator
INFO[0035] deleting packages pkg=advanced-cluster-management
INFO[0035] packages: [advanced-cluster-management] pkg=advanced-cluster-management
...
...
INFO[0064] writing dockerfile: ./index.Dockerfile3335274021 packages="[elasticsearch-operator]"
INFO[0064] running podman build packages="[elasticsearch-operator]"
INFO[0064] [podman build --format docker -f ./index.Dockerfile3335274021 -t installer.ocp.jerome.co.za:8443/redhat/redhat-operator-index:v4.10 .] packages="[elasticsearch-operator]"… and push it to our registry…
# podman push installer.ocp.jerome.co.za:8443/redhat/redhat-operator-index:v4.10 --authfile pull-secret.txt --tls-verify=false
Getting image source signatures
Copying blob fd961d6ccfde done
Copying blob fa9b438e48f4 done
Copying blob 821d9ad67467 done
Copying blob fcc6c36b5caa done
Copying blob 1baf2ef4355a done
Copying blob 1b2d573e2d84 done
Copying config 39f60df574 done
Writing manifest to image destination
Storing signatures… and grab the images for it (our specified operator/s).
Note: we filter via our os/architecture — so as to avoid grabbing things like ARM images, when we only need the x86–64 one’s.
Saving you time… space… and money!
# oc adm catalog mirror installer.ocp.jerome.co.za:8443/redhat/redhat-operator-index:v4.10 installer.ocp.jerome.co.za:8443/redhat/redhat-operator-index:v4.10 --index-filter-by-os='linux/amd64' --insecure=true
Warning: the default reading order of registry auth file will be changed from "${HOME}/.docker/config.json" to podman registry config locations in the future version. "${HOME}/.docker/config.json" is deprecated, but can still be used for storing credentials as a fallback. See https://github.com/containers/image/blob/main/docs/containers-auth.json.5.md for the order of podman registry config locations.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!! DEPRECATION NOTICE:
!! Sqlite-based catalogs are deprecated. Support for them will be removed in a
!! future release. Please migrate your catalog workflows to the new file-based
!! catalog format.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
src image has index label for database path: /database/index.db
using index path mapping: /database/index.db:/tmp/3843617184
Warning: the default reading order of registry auth file will be changed from "${HOME}/.docker/config.json" to podman registry config locations in the future version. "${HOME}/.docker/config.json" is deprecated, but can still be used for storing credentials as a fallback. See https://github.com/containers/image/blob/main/docs/containers-auth.json.5.md for the order of podman registry config locations.
wrote database to /tmp/3843617184
using database at: /tmp/3843617184/index.db
Warning: the default reading order of registry auth file will be changed from "${HOME}/.docker/config.json" to podman registry config locations in the future version. "${HOME}/.docker/config.json" is deprecated, but can still be used for storing credentials as a fallback. See https://github.com/containers/image/blob/main/docs/containers-auth.json.5.md for the order of podman registry config locations.
installer.ocp.jerome.co.za:8443/
redhat/redhat-operator-index-openshift-logging-elasticsearch-operator-bundle
blobs:
registry.redhat.io/openshift-logging/elasticsearch-operator-bundle sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 32B
registry.redhat.io/openshift-logging/elasticsearch-operator-bundle sha256:813d94d47b0041c4da11bdff39c7a386c28837025392bbd4afc451d1154f56cf 3.659KiB
registry.redhat.io/openshift-logging/elasticsearch-operator-bundle sha256:7e6b2a53ee3a19cf45f9dc33e8ab096b52d444c4fd20636c0213ccfcbf71282a 5.514KiB
registry.redhat.io/openshift-logging/elasticsearch-operator-bundle sha256:4f024100c496545d43a8ad48861fa74fba413402bfc7ed4433207fa3c9f22add 5.541KiB
...
...
info: Mirroring completed in 8m6.47s (4.901MB/s)
no digest mapping available for installer.ocp.jerome.co.za:8443/redhat/redhat-operator-index:v4.10, skip writing to ImageContentSourcePolicy
wrote mirroring manifests to manifests-redhat-operator-index-1670413348
deleted dir /tmp/3668626965Now we’ll apply the catalogSource policy and the imageContentSourcePolicy so our cluster looks for operators within our local mirror-registry and NOT the internet.
# cd manifests-redhat-operator-index-1670413348/
# oc apply -f catalogSource.yaml
catalogsource.operators.coreos.com/redhat-operator-index created
# oc apply -f imageContentSourcePolicy.yaml
imagecontentsourcepolicy.operator.openshift.io/redhat-operator-index-0 createdWith this done, our cluster should now be able to see the new elasticsearch-operator.
Let’s install it.
Then double check it’s listed as installed under our “Installed Operators”
OR the CLI.
# oc get all -n openshift-operators-redhat
NAME READY STATUS RESTARTS AGE
pod/elasticsearch-operator-5b5cff9f78-hxq4p 2/2 Running 0 3m33s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/elasticsearch-operator-metrics ClusterIP 172.30.10.105 <none> 8443/TCP 3m34s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/elasticsearch-operator 1/1 1 1 3m33s
NAME DESIRED CURRENT READY AGE
replicaset.apps/elasticsearch-operator-5b5cff9f78 1 1 1 3m33sNowNow you know what it takes to install a “disconnected” cluster — in PART3 we’ll discuss maintenance — what it takes to upgrade a disconnected cluster with additional operators AND an OpenShift version upgrade.
