Create your own custom and authenticated APT repository
Internet is full of resources on how to create an APT local repository and how to expose it via HTTP. Unfortunately, APT changed quite a lot since these tutorials and howtos have been written: most of them do not work out of the box. Examples of changes are SHA1 removal, which broke many many repos, and turning on authenticated repositories by default since APT 1.1.
This howto works with Ubuntu 16.04.4 (latest LTS at the time of writing) and uses SHA256 signatures. It will help setting up GPG keys, draft a script to update your repo and provide many crunchy details about APT internals.
APT Repository Layout
The ultimate goal of the repository is to be exposed via HTTP (using apache httpd), so the best place we can store the repo is inside the DocumentRoot of httpd, which defaults to /var/www/html/
. We'll use a subfolder here, namely /var/www/html/repo
.
The listing below shows the organisation inside this repository:
/var/www/html/repo/
|-- amd64/ ← where arch=amd64 *.deb packages will go
| |-- package1-version1.deb
| └-- ... more deb files here ...
|-- InRelease ← "in-file-signed" Release file (see below)
|-- KEY.gpg ← gpg key signing the repository and packages
|-- Packages ← description of repo's packages
|-- Packages.gz ← gzip'ed version of the Package file above
|-- Release ← metadata (file listing and checksum) of the current distribution (i.e. the current repo in our case)
|-- Release.gpg ← gpg signature of the file above
If you want more details, the complete repository format specification can be found here: https://wiki.debian.org/DebianRepository/Format
Setup GPG Key
The first step, also a one-off step, is to create a GPG key. This is rather simple, but you might be surprised that default gpg key still uses SHA1 and some extra work is needed to avoid this.
The following section assumes gnupg is installed.
Gnupg configuration
The~/.gnupg/gpg.conf
file needs some tweaking:
mkdir ~/.gnupg
echo "cert-digest-algo SHA256" >> ~/.gnupg/gpg.conf
echo "digest-algo SHA256" >> ~/.gnupg/gpg.conf
Prompt-less generation of a GPG key
This part explains how to generate a GPG key without any prompted question. (But please keep in mind that this is a bad idea.)
The following snippet assumes that $KEYNAME and $EMAIL variables are set.
KEYNAME=dpkg1
EMAIL=${KEYNAME}@mydomain.ext
Generate a configuration file used to generate the key.
cat > $KEYNAME.batch <<EOF
%echo Generating a standard key
Key-Type: RSA
Key-Length: 4096
Subkey-Length: 4096
Name-Real: ${KEYNAME}
Name-Email: ${EMAIL}
Expire-Date: 0
%pubring ${KEYNAME}.pub
%secring ${KEYNAME}.key
# Do a commit here, so that we can later print "done" :-)
%commit
%echo done
EOF
Generate the key. Two files will be created by the following line, respectively ${KEYNAME}.pub
and ${KEYNAME}.key:
gpg --batch --gen-key $KEYNAME.batch
Now you can import the key, using the following commands:
gpg --no-default-keyring --secret-keyring ${KEYNAME}.key --keyring ${KEYNAME}.pub --list-secret-keysgpg --import ${KEYNAME}.key
Retrieving the key id
The next step is to retrieve the ID of your key to be able to export it in the right format. The following command will allow you to retrieve the ID of the key:
gpg ${KEYNAME}.key
The key id is the hexstring after 4096R/, i.e. EAE1F4AA
in the example below:
sec 4096R/EAE1F4AA 2018-03-13 dpkg1 <dpkg1@mydomain.ext>
Export the key and store it within the APT repo to be imported by your repo users:
gpg --output /var/www/html/repo/KEY.gpg --armor --export $KEYID
You’re all good, you have a key referred as ${KEYNAME} created, exported and usable.
For the curious, more details about GPG and GPG migration are provided by the ubuntu Security team here: https://wiki.ubuntu.com/SecurityTeam/GPGMigration
APT Local Repository
The next task in our journey is to create a local APT repository. For this we’ll need to install a bunch of packages and generate the repo-specific files like Packages and Release.
Setup local repo
Let’s install a bunch of packages:
apt install apache2 dpkg-dev dpkg-sig
The apache2 package should have created /var/www/html/, and we’ll create our dedicated directory where we’ll store the packages and metadata:
mkdir -p /var/www/html/repo/
This was easy. Next, create the metadata files.
Let’s move to our repository and assume all the later commands are run from this working directory:
cd /var/www/html/repo/
Packages
There are tools to generate this Packages, mostly apt-ftparchive and dpkg-scanpackages. We’ll use here apt-ftparchive:
apt-ftparchive --arch amd64 packages amd64 > Packages
A gzip’ed version of the file can also be provided to save a bit of bandwidth (please ensure -k is used, i.e. both Packages and Packages.gz are present is the repository):
gzip -k -f Packages
Release
For building the Release file, we’ll use again apt-ftparchive:
apt-ftparchive release . > Release
Now we can sign the Release file, in two version: a stand-alone Release.gpg file and a InRelease which embbeded the signature in the file.
The first rm statement is to ensure prompt-less command if the file already exists, i.e. this can be used in a script run every time you add a new deb packages in your repo.
Generate Release.gpg
file
rm -fr Release.gpg; gpg --default-key ${KEYNAME} -abs -o Release.gpg Release
Generate InRelease
file
rm -fr InRelease; gpg --default-key ${KEYNAME} --clearsign -o InRelease Release
Done! You know have a fully authenticated APT repository, which is exposed through HTTP. You’re no longer forced to use [trusted=yes]
in your apt source list!
Using your APT repository from another server
Now you have a working and authenticated APT repository exposed through HTTP, you can configure apt-clients to use your repository via the following set of configuration.
We assume here that the HOST variable is set, for instance:
HOST=1.2.3.4echo "deb http://${HOST}/repo/ /" > /etc/apt/sources.list.d/internal.list
And importing the GPG key
wget -q -O - http://${HOST}/repo/KEY.gpg | apt-key add -
Try it, i.e. running apt update
should not throw any error:
apt update
Bonus: Signing a deb package
With the GPG setup detailed above, signing a deb package is straightforward. Still assuming the variable ${KEYNAME}
is set accordingly:
dpkg-sig -k ${KEYNAME} --sign repo my-package.deb
Your deb file is now signed. You can verify this using
dpkg-sig -c my-package.deb