The Wonders of GNU and OCI (docker) Containers
Creating a pipeline to download, compile and create an OCI compatible container.
I came across some an old odd file format with extension .ccd belonging to the now defunct Clone CD application, that was all the rage in the early 2000s. The program was originally created by a company called *Slysoft* and was replaced by Redfox when targeted by various entities that opposed the bypassing of Digital Rights Management (DRM) when copying or in general the copying of CDs.
Most of the tools to convert between .ccd format are made for Windows but there are a number of open source tools of GNU/Linux environments such as ccd2isoand ccd2cue . The latter has no packages for Ubuntu and therefore an opportunity to showcase the wonders of both Containers and Open Source software!
Manual Compilation
On a Ubuntu 20.04 host using x86_64 architecture, download the source files, install typical build tools and a C compiler such as gcc:
# Create suitable directory
mkdir -p source/ccd2cue# Change into directory
cd source/ccd2cue# Download source files
wget https://ftp.gnu.org/gnu/ccd2cue/ccd2cue-0.5.tar.gz# Untar
tar xvzf ccd2cue-0.5.tar.gz# Change into source directory
cd ccd2cue-0.5
The installation instructions can be summarized as:
# Check configuration
./configure# Make artifacts
make# Install (using privileged account)
# if installing in default or secure
# path. On Ubuntu can use sudo:
sudo make install
Running ./configure will show that a C compiler is missing as are essential build tools such as make. This can be rectified using:
# Install dependencies
sudo apt install build-essential gccCheck configuration:
# Check configuration
./configurechecking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports nested variables... (cached) yes
checking whether NLS is requested... yes
checking for msgfmt... no
checking for gmsgfmt... :
checking for xgettext... no
checking for msgmerge... no
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking for ld used by GCC... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for shared library run path origin... done
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for CFPreferencesCopyAppValue... no
checking for CFLocaleCopyCurrent... no
checking for GNU gettext in libc... yes
checking whether to use NLS... yes
checking where the gettext function comes from... libc
checking whether ln -s works... yes
checking for grep that handles long lines and -e... (cached) /usr/bin/grep
checking for a sed that does not truncate output... /usr/bin/sed
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking whether gcc understands -c and -o together... (cached) yes
checking dependency style of gcc... (cached) gcc3
configure: checking programs needed in order to build documentation
checking for makeinfo... no
configure: WARNING: GNU Texinfo not found. It is required at least version 5.1.
You can find it at https://gnu.org/software/texinfo/.
It will not be possible to rebuild the Info documentation
nor generate any other documentation format.
checking for help2man... no
checking for doxygen... no
configure: WARNING: doxygen not found - will not generate any doxygen documentation
checking for perl... /usr/bin/perl
configure: Checking meta-information
checking available translations... da de uk vi
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating po/Makefile.in
config.status: creating po/LINGUAS
config.status: creating src/Makefile
config.status: creating doc/Makefile
config.status: creating doc/Doxyfile
config.status: creating doc/man/Makefile
config.status: creating doc/release/release.texi
config.status: creating config.h
config.status: config.h is unchanged
config.status: executing po-directories commands
config.status: creating po/POTFILES
config.status: creating po/Makefile
config.status: executing depfiles commands## ---------------------- ##
## GNU Make Targets Table ##
## ---------------------- ##Build:all Compile entire program.info, html, pdf, ps, dvi
Build documentation in the respective format.doxygen-doc Build source-code documentation.
check Run test suite.tags, ctags
Build respective tags table file.Installation:install-exec Install architecture-dependent files.
install-data Install architecture-independent files.
install Same as `install-exec' followed by `install-data'.
install-strip Same as `install' but strip executables.install-{info,html,pdf,ps,dvi}
Install documentation in the respective format.install-man Install man pages.
installdirs Create the installation directory tree.
installcheck Perform installation tests.
uninstall Delete installed files.Cleaning:mostlyclean Delete files that `make' built and usually
one would like to rebuild.
clean Same as `mostlyclean' but delete also
any file `make' built but was ignored by
`mostlyclean'.
distclean Same as `clean' but delete also files
generated by the configuration script.
maintainer-clean Same as `distclean' but delete also
files maintainers built.Distribution:distdir Create distribution directory.
dist Same as `distdir' but package the distribution
directory into a tarball.
distcheck Same as `dist' but also check generated
tarball for correctness.Maintenance:These can only be made from VCS checkouts.homepage Generate package's homepage.
distgnu Same as `distcheck' but also sign and upload the
resulting tarball to the GNU ftp site.
announcegnu Send release announcement to 'info-gnu@gnu.org',
'ccd2cue@gnu.org' and
'coordinator@translationproject.org'
fetchpo Fetch the latest PO translation files from the
Translation Project.
Make the artifact:
# Build
makemake all-recursive
make[1]: Entering directory '/home/helix/source/ccd2cue/ccd2cue-0.5'
Making all in po
make[2]: Entering directory '/home/helix/source/ccd2cue/ccd2cue-0.5/po'
make[2]: Leaving directory '/home/helix/source/ccd2cue/ccd2cue-0.5/po'
Making all in src
make[2]: Entering directory '/home/helix/source/ccd2cue/ccd2cue-0.5/src'
CC ccd2cue.o
CC memory.o
CC io.o
CC file.o
CC array.o
CC errors.o
CC crc.o
CC ccd.o
ccd.c: In function ‘stream2ccd’:
ccd.c:362:33: warning: format ‘%a’ expects argument of type ‘float *’, but argument 3 has type ‘char **’ [-Wformat=]
362 | if (sscanf (line, " FLAGS = %a[a-zA-Z0-9 ] ", &ccd->TRACK[TRACK].FLAGS) == 1)
| ~^ ~~~~~~~~~~~~~~~~~~~~~~~~
| | |
| float * char **
CC cue.o
CC cdt.o
CC convert.o
CCLD ccd2cue
make[2]: Leaving directory '/home/helix/source/ccd2cue/ccd2cue-0.5/src'
Making all in doc
make[2]: Entering directory '/home/helix/source/ccd2cue/ccd2cue-0.5/doc'
Making all in man
make[3]: Entering directory '/home/helix/source/ccd2cue/ccd2cue-0.5/doc/man'
make[3]: Nothing to be done for 'all'.
make[3]: Leaving directory '/home/helix/source/ccd2cue/ccd2cue-0.5/doc/man'
make[3]: Entering directory '/home/helix/source/ccd2cue/ccd2cue-0.5/doc'
make[3]: Nothing to be done for 'all-am'.
make[3]: Leaving directory '/home/helix/source/ccd2cue/ccd2cue-0.5/doc'
make[2]: Leaving directory '/home/helix/source/ccd2cue/ccd2cue-0.5/doc'
make[2]: Entering directory '/home/helix/source/ccd2cue/ccd2cue-0.5'
make[2]: Leaving directory '/home/helix/source/ccd2cue/ccd2cue-0.5'
make[1]: Leaving directory '/home/helix/source/ccd2cue/ccd2cue-0.5'
The final step is to install onto the system:
# Install into default location
sudo make installMaking install in po
make[1]: Entering directory '/home/helix/source/ccd2cue/ccd2cue-0.5/po'
installing da.gmo as /usr/local/share/locale/da/LC_MESSAGES/ccd2cue.mo
installing de.gmo as /usr/local/share/locale/de/LC_MESSAGES/ccd2cue.mo
installing uk.gmo as /usr/local/share/locale/uk/LC_MESSAGES/ccd2cue.mo
installing vi.gmo as /usr/local/share/locale/vi/LC_MESSAGES/ccd2cue.mo
if test "ccd2cue" = "gettext-tools"; then \
/usr/bin/mkdir -p /usr/local/share/gettext/po; \
for file in Makefile.in.in remove-potcdate.sin quot.sed boldquot.sed en@quot.header en@boldquot.header insert-header.sin Rules-quot Makevars.template; do \
/usr/bin/install -c -m 644 ./$file \
/usr/local/share/gettext/po/$file; \
done; \
for file in Makevars; do \
rm -f /usr/local/share/gettext/po/$file; \
done; \
else \
: ; \
fi
make[1]: Leaving directory '/home/helix/source/ccd2cue/ccd2cue-0.5/po'
Making install in src
make[1]: Entering directory '/home/helix/source/ccd2cue/ccd2cue-0.5/src'
make[2]: Entering directory '/home/helix/source/ccd2cue/ccd2cue-0.5/src'
/usr/bin/mkdir -p '/usr/local/bin'
/usr/bin/install -c ccd2cue '/usr/local/bin'
make[2]: Nothing to be done for 'install-data-am'.
make[2]: Leaving directory '/home/helix/source/ccd2cue/ccd2cue-0.5/src'
make[1]: Leaving directory '/home/helix/source/ccd2cue/ccd2cue-0.5/src'
Making install in doc
make[1]: Entering directory '/home/helix/source/ccd2cue/ccd2cue-0.5/doc'
Making install in man
make[2]: Entering directory '/home/helix/source/ccd2cue/ccd2cue-0.5/doc/man'
make[3]: Entering directory '/home/helix/source/ccd2cue/ccd2cue-0.5/doc/man'
make[3]: Nothing to be done for 'install-exec-am'.
/usr/bin/mkdir -p '/usr/local/share/man/man1'
/usr/bin/install -c -m 644 ccd2cue.1 '/usr/local/share/man/man1'
make[3]: Leaving directory '/home/helix/source/ccd2cue/ccd2cue-0.5/doc/man'
make[2]: Leaving directory '/home/helix/source/ccd2cue/ccd2cue-0.5/doc/man'
make[2]: Entering directory '/home/helix/source/ccd2cue/ccd2cue-0.5/doc'
make[3]: Entering directory '/home/helix/source/ccd2cue/ccd2cue-0.5/doc'
make[3]: Nothing to be done for 'install-exec-am'.
/usr/bin/mkdir -p '/usr/local/share/info'
/usr/bin/install -c -m 644 ./ccd2cue.info '/usr/local/share/info'
make[3]: Leaving directory '/home/helix/source/ccd2cue/ccd2cue-0.5/doc'
make[2]: Leaving directory '/home/helix/source/ccd2cue/ccd2cue-0.5/doc'
make[1]: Leaving directory '/home/helix/source/ccd2cue/ccd2cue-0.5/doc'
make[1]: Entering directory '/home/helix/source/ccd2cue/ccd2cue-0.5'
make[2]: Entering directory '/home/helix/source/ccd2cue/ccd2cue-0.5'
make[2]: Nothing to be done for 'install-exec-am'.
make[2]: Nothing to be done for 'install-data-am'.
make[2]: Leaving directory '/home/helix/source/ccd2cue/ccd2cue-0.5'
make[1]: Leaving directory '/home/helix/source/ccd2cue/ccd2cue-0.5'
Test the application:
# Test application
ccd2cue --versionccd2cue (GNU ccd2cue) 0.5Copyright (C) 2010, 2013, 2014, 2015 Bruno Fe'lix Rezende Ribeiro <oitofelix@gnu.org>License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.Written by Bruno Fe'lix Rezende Ribeiro.
OCI or Docker Container
The alternative approach is to build it, compile it and install it within a container using the following ContainerFile (similar to a DockerFile).
The Open Container Initiative (OCI) is a standard intended to replace the
dockercontainer format with a consistent standard.
Although the docker application could be used to build and manage containers, tools such as buildah and podman are increasingly lighter, more agile and standards compliant. The following file summarizes the manual steps:
FROM ubuntu:20.04LABEL "org.opencontainers.image.authors"="miah0x41"# Install dependencies
RUN apt update && apt install -y \
build-essential \
gcc \
wget \
&& rm -rf /var/lib/apt/lists/*# Create folder
RUN mkdir -p /source/ccd2cue# Change working directory
WORKDIR /source/ccd2cue# Download source code
RUN wget https://ftp.gnu.org/gnu/ccd2cue/ccd2cue-0.5.tar.gz \
&& tar xvzf ccd2cue-0.5.tar.gz \
&& rm ccd2cue-0.5.tar.gz# Change into source directory
WORKDIR /source/ccd2cue/ccd2cue-0.5# Configure, make and install
RUN ./configure \
&& make \
&& make install# Set default command
ENTRYPOINT ["/usr/local/bin/ccd2cue"]
To create the image use buildah with the following command:
# Build image
buildah bud -f ContainerFile -t miah0x41/ccd2cue:0.5-0WARN[0000] Failed to decode the keys ["machine"] from "/usr/share/containers/containers.conf".
WARN[0000] Failed to decode the keys ["machine"] from "/usr/share/containers/containers.conf".
STEP 1/9: FROM ubuntu:20.04
STEP 2/9: LABEL "org.opencontainers.image.authors"="miah0x41"
STEP 3/9: RUN apt update && apt install -y build-essential gcc wget && rm -rf /var/lib/apt/lists/*
WARN[0000] Failed to decode the keys ["machine"] from "/usr/share/containers/containers.conf".WARNING: apt does not have a stable CLI interface. Use with caution in scripts.Get:1 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]
Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Get:4 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1205 kB]
Get:5 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB]
<snip>
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
STEP 4/9: RUN mkdir -p /source/ccd2cue
WARN[0000] Failed to decode the keys ["machine"] from "/usr/share/containers/containers.conf".
STEP 5/9: WORKDIR /source/ccd2cue
STEP 6/9: RUN wget https://ftp.gnu.org/gnu/ccd2cue/ccd2cue-0.5.tar.gz && tar xvzf ccd2cue-0.5.tar.gz && rm ccd2cue-0.5.tar.gz
WARN[0000] Failed to decode the keys ["machine"] from "/usr/share/containers/containers.conf".
--2022-05-30 21:24:37-- https://ftp.gnu.org/gnu/ccd2cue/ccd2cue-0.5.tar.gz
Resolving ftp.gnu.org (ftp.gnu.org)... 2001:470:142:3::b, 209.51.188.20
Connecting to ftp.gnu.org (ftp.gnu.org)|2001:470:142:3::b|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 626071 (611K) [application/x-gzip]
Saving to: 'ccd2cue-0.5.tar.gz'0K .......... .......... .......... .......... .......... 8% 257K 2s
50K .......... .......... .......... .......... .......... 16% 516K 1s
100K .......... .......... .......... .......... .......... 24% 94.9M 1s
150K .......... .......... .......... .......... .......... 32% 518K 1s
200K .......... .......... .......... .......... .......... 40% 77.5M 1s
250K .......... .......... .......... .......... .......... 49% 94.8M 0s
300K .......... .......... .......... .......... .......... 57% 111M 0s
350K .......... .......... .......... .......... .......... 65% 523K 0s
400K .......... .......... .......... .......... .......... 73% 39.1M 0s
450K .......... .......... .......... .......... .......... 81% 118M 0s
500K .......... .......... .......... .......... .......... 89% 110M 0s
550K .......... .......... .......... .......... .......... 98% 115M 0s
600K .......... . 100% 65.0M=0.5s2022-05-30 21:24:38 (1.22 MB/s) - 'ccd2cue-0.5.tar.gz' saved [626071/626071]<snip>
ccd2cue-0.5/INSTALL
ccd2cue-0.5/TODO
STEP 7/9: WORKDIR /source/ccd2cue/ccd2cue-0.5
STEP 8/9: RUN ./configure && make && make install
WARN[0000] Failed to decode the keys ["machine"] from "/usr/share/containers/containers.conf".
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
<snip>
STEP 9/9: ENTRYPOINT ["/usr/local/bin/ccd2cue"]
COMMIT miah0x41/ccd2cue:0.5
Getting image source signatures
Copying blob bf8cedc62fb3 skipped: already exists
Copying blob 3bd397d87e45 done
Copying config 9b3c5ca6c6 done
Writing manifest to image destination
Storing signatures
--> 9b3c5ca6c6c
Successfully tagged localhost/miah0x41/ccd2cue:0.5
9b3c5ca6c6c4b8518c4259cf73baeb468a6d72f8f4dd74ba52239f5b5b8265a3
Test the container with the --version command:
# Check version
podman run --rm -it miah0x41/ccd2cue --versionccd2cue (GNU ccd2cue) 0.5Copyright (C) 2010, 2013, 2014, 2015 Bruno Fe'lix Rezende Ribeiro <oitofelix@gnu.org>License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.Written by Bruno Fe'lix Rezende Ribeiro.
Whilst this creates a portable container for the ccd2cue program, it’s not widely accessible. This can of course be solved by uploading to Docker Hub manually, however it’s better for transparency and the wider community to use a public repo.
GitHub Actions
Instead of manually running buildah on a local machine, we can leverage GitHub actions using predefined templates. The first step is to host the ContainerFile in a repository such as miah0x41/ccd2cue . The top bar contains the available actions:
Under Actions, click on “New Workflow”, and under “Continuous Integration” can use the “Docker Image” template.
However, buried in the documentation there is a build-and-push action, which is more suited. The following action file is used in this example:
There are many parts to this action; fundamentally the on: key defines when the action is run both automatically and with workflow_dispatch manually as well. Environmental variables are used to define the registry (as GitHub’s own) as well as the permissions for the Github token (used to authenticate with the registry). In this instance the name of the registry ccd2cue is also the name of the image.
The key steps in the job are to checkout the repository, followed by loggin into the registry, which uses a combination of Github variables (such as actor), environmental variables and Github secrets. The next step extracts the meta data from the image (to be honest not sure what this step exactly does!).
To utilise the date formatted in seconds as part of the tag the bash command is stored within an environmental variable called TAG . This is used in the final step where the context is set for the build process. The path is identified as the root of the repo (i.e. .) and the tag is formed from a combination of environmental variables representing registry, image name and tag.
Committing the changes will trigger the build process:
Under packages in the repo home page a link to the image is provided:
Clicking on it provides the image details:
Although there are more steps involved, once an GitHub action has been generated it can be copied across to other projects.
Conclusion
Starting with an obscure package unavailable in Ubuntu 20.04, this guide shows how it could be compiled locally, prior to migrating within a container and then automating that process.
Attribution
All gists , notebooks and terminal casts are by the author. All of the artwork is based on assets explicitly CC0, Public Domain license or SIL OFL and is therefore non-infringing. Theme is inspired by and based on my favourite vim theme: Gruvbox.
