Secure Software Updates via TUF — Part 2
The Update Framework: Metadata File Format and Repository Layout
The Update Framework (TUF) is an open source framework that will help you secure your software update delivery system holistically, covering areas that you may not have given much thought from the point of view of security.
TUF secures the software update delivery system using mechanisms such as roles, their signatures (PKI), threshold number of signatures, file hashes, and file size. All this information is provided in the form of metadata files which are version numbered and have an expiration date.
Here’s part one of my blog post on TUF that covers: why securing the entire software update delivery system is needed, how TUF works to provide this security, the design principles that underpin TUF, and its structure.
This blog post, which is part two, focuses on the metadata file format of the five top-level roles, and on the repository layout generated by the reference implementation.
Metadata files can be in any data format with the caveat being that all the fields mentioned in the TUF specification are included and interpreted correctly. The TUF specification uses the JSON format for its examples and so shall I in the example snippets below.
The five top-level roles and their corresponding metadata (json) files are: Root (root.json), Targets (targets.json), Snapshot (snapshot.json), Timestamp (timestamp.json), and Mirrors (mirrors.json). Only the Mirrors role is optional.
The examples below list only the necessary fields, but you also have the option of adding custom fields into these metadata files. For a comprehensive list of fields, do read the TUF Specification on GitHub.
In the examples below, capitalized words will be replaced with actual values when the metadata file is generated, for example, ROOT_KEY_ID will be replaced with the actual key identifier value, and ROOT’S_SIGNATURE will be replaced with the actual signature.
Let’s dive right in and see what these metadata files are made of.
Metadata file format
root.json
{
"signatures": [
{
"keyid": "ROOT_KEY_ID",
"sig": "ROOT'S_SIGNATURE"
}
],
"signed": {
"_type": "root",
"spec_version": "TUF_SPEC_VERSION",
"consistent_snapshot": CONSISTENT_SNAPSHOT_VALUE,
"expires": "ROOT.JSON_EXPIRY_DATE_TIME",
"keys": {
"KEY_ID": {
"keytype": "KEY_TYPE",
"scheme": "SIGNATURE_SCHEME",
"keyval": {
"public": "PUBLIC_KEY"
}
},
},
"roles": {
"root": {
"keyids": [
"ROOT_KEY_ID"
],
"threshold": THRESHOLD_VALUE
},
"snapshot": {
"keyids": [
"SNAPSHOT_KEY_ID"
],
"threshold": THRESHOLD_VALUE
},
"targets": {
"keyids": [
"TARGET_KEY_ID"
],
"threshold": THRESHOLD_VALUE
},
"timestamp": {
"keyids": [
"TIMESTAMP_KEY_ID"of the
],
"threshold": THRESHOLD_VALUE
}
},
"version": ROOT.JSON_VERSION_NUM
}
}root.json starts off by listing its own key ID and signature to establish authenticity, and then goes on to list the TUF specification version used, which ensures that root.json’s metadata format matches with the format stated in that version of the specification.
Next in line is the consistent snapshot field, which is a boolean value that informs whether the repository supports consistent snapshots or not. If it’s enabled then the framework ensures that it downloads only those files whose filename starts with a version number (variable filename), for example 6.root.json. If it’s disabled then the framework does not bother about file names having to start with a version number and instead looks for a file called root.json and downloads it (fixed filename).
After that comes root.json’s expiration date and time, followed by the list of keys being used in the repository. Each key lists its cryptographic algorithm type (rsa, ed25519, or ecdsa-sha2-nistp256), signature scheme within the type (rsassa-pss-sha256, ed25519, or ecdsa-sha2-nistp256), and the public key.
The key types and signature schemes listed above in brackets are just examples. You’re free to choose any cryptographic library, algorithm type and signature scheme that suits your purpose.
Next in the file is the assignment of keys to roles such as root, targets, snapshot, and timestamp; all four need to be mandatorily listed (assigned). Mirrors role is optional, and if not listed means that the mirrors list (mirrors.json) will not be signed. Threshold value, which is the minimum number of signatures required from that particular role, is also listed along with the key assignment.
The last field is root.json’s version number to help figure out if a new version of this file has been posted or not. It also helps prevent rollback attack by ensuring that a version less than the currently trusted one is never downloaded and used.
targets.json
{
"signatures": [
{
"keyid": "TARGETS_KEY_ID",
"sig": "TARGET'S_SIGNATURE"
}
],
"signed": {
"_type": "targets",
"spec_version": "TUF_SPEC_VERSION",
"delegations": {
"keys": {
"KEY_ID": {
"keytype": "KEY_TYPE",
"scheme": "SIGNATURE_SCHEME",
"keyval": {
"public": "PUBLIC_KEY"
}
}
},
"roles": [
{
"keyids": [
"KEY_ID"
],
"name": "DELEGATED_ROLE_NAME",
"paths": [
"TARGET_FILE_PATH"
],
"threshold": THRESHOLD_VALUE
}
]
},
"expires": "TARGETS.JSON_EXPIRY_DATE_TIME",
"targets": {
"TARGET_FILE_NAME": {
"hashes": {
"sha256": "TARGET_FILE_SHA256_HASH",
"sha512": "TARGET_FILE_SHA512_HASH"
},
"length": TARGET_FILE_LENGTH_BYTES
},
},
"version": TARGETS.JSON_VERSION_NUM
}
}targets.json starts off similar to root.json, with a signature. An interesting thing that targets can do is delegations, where it delegates trust to another role to sign off on a target file. Signing off on a target file means that a metadata file signed by the delegated role is generated for that target file; the same function that the targets role performs unless there has been a delegation.
The format for delegations is the same as in root.json, where the keys are listed along with their accompanying data, and then assigned to the delegated roles. Full or partial delegations can be done, where the delegated role is either responsible for signing some of the target files or all of them.
The assignment part also has a ‘paths’ field which specifies the path to the target file that has been delegated to this particular role.
Targets section lists the target files along with their hashes (one or more), and file size (length). TARGET_FILE_NAME can just be a filename or a path to the file depending on the location of the target file.
Revocation of keys is as easy as either removing the keys from the metadata file or replacing them with another key, in the next version of the file.
snapshot.json
{
"signatures": [
{
"keyid": "KEY_ID",
"sig": "SNAPSHOT'S_SIGNATURE"
}
],
"signed": {
"_type": "snapshot",
"spec_version": "TUF_SPEC_VERSION",
"expires": "SNAPSHOT.JSON_EXPIRY_DATE_TIME",
"meta": {
"targets.json": {
"version": TARGETS.JSON_VERSION_NUM
},
"TARGET_FILENAME1.json": {
"version": TARGET_FILENAME1_VERSION_NUM,
"hashes": {
"sha256": "TARGET_FILE_SHA256_HASH",
"sha512": "TARGET_FILE_SHA512_HASH"
}
},
"TARGET_FILENAME2.json": {
"version": TARGET_FILENAME2_VERSION_NUM,
"length": TARGET_FILE_LENGTH_BYTES,
"hashes": {
"sha256": "TARGET_FILE_SHA256_HASH"
}
}
},
"version": SNAPSHOT.JSON_VERSION_NUM
}
}snapshot.json provides a view (snapshot) of the latest set of targets metadata files. It does this using fields such as ‘meta’ which lists metadata information of targets.json and the other targets metadata files.
The actual target files will all have corresponding metadata files vouching for their authenticity. snapshot.json contains metadata information about each of these metadata files. Information such as, version number, file size, and file hashes.
This information is used by clients to check which files are new and need to be downloaded. The metadata information is also used to verify the authenticity of the files.
timestamp.json
{
"signatures": [
{
"keyid": "KEY_ID",
"sig": "TIMESTAMP'S_SIGNATURE"
}
],
"signed": {
"_type": "timestamp",
"spec_version": "TUF_SPEC_VERSION",
"expires": "TIMESTAMP.JSON_EXPIRY_DATE_TIME",
"meta": {
"snapshot.json": {
"hashes": {
"sha256": "SNAPSHOT.JSON_SHA256_HASH"
},
"length": SNAPSHOT.JSON_LENGTH,
"version": SNAPSHOT.JSON_VERSION
}
},
"version": TIMESTAMP.JSON_VERSION_NUM
}
}timestamp.json deals exclusively with information about snapshot.json. It also has a ‘meta’ field that lists metadata information about snapshot.json. Information such as, version number, file size, and file hashes.
This information is used by clients to check if a new snapshot.json has been released, and also to verify snapshot.json’s authenticity.
mirrors.json
{
"signatures": [
{
"keyid": "KEY_ID",
"sig": "MIRROR'S_SIGNATURE"
}
],
"signed": {
"_type": "mirrors",
"spec_version": “TUF_SPEC_VERSION”,
"expires": “MIRRORS.JSON_EXPIRY_DATE_TIME”,
"mirrors": [
{ "urlbase": “MIRRORS_BASE_URL”,
"metapath": “METADATA_FILES_PATH”,
"targetspath": “TARGET_FILES_PATH”,
"metacontent": [METADATA_FILES_AVAILABLE],
"targetscontent": [TARGET_FILES_AVAILABLE],
},
]
"version": MIRRORS.JSON_VERSION_NUM
}
}mirrors.json lists the repository’s mirrors and the content — metadata files and target files — that is available with them. For example, if the mirror repository is a partial mirror hosting only some of the files as compared to the main repository, then the metadata files and target files that have been made available in that partial mirror can be listed here in mirrors.json file.
If it’s a full mirror then all the metadata files and target files from the main repository can be made available in the mirror and in the mirrors.json file.
mirrors.json has fields such as ‘urlbase’ which specifies the URL of the mirror. ‘metapath’ specifies the path to the metadata files, and ‘targetspath’ specifies the path to the target files. ‘metacontent’ and ‘targetscontent’ fields specify the metadata files and target files that are available with that mirror.
The software update system that is using TUF can also choose to hardcode the mirror information and thus forego using mirrors.json.
Repository Layout
TUF and the software update system that it serves have to work with repositories and their mirrors. Each repository or mirror has its own set of roles, keys, metadata files and target files. Keys should not be shared across repositories or mirrors because a compromise at one can lead to a compromise across all repositories or mirrors that share the keys.
TUF needs the repositories to be arranged in a certain way. This arrangement (layout) helps assign roles to a specific part of the repository for which they can sign, and it also helps partial mirrors to easily replicate only some of the files.
The metadata files and their layout has to conform to TUF specifications, whereas the target files and their layout need not. The application that is using TUF can decide on the naming convention and the directory structure of the target files.
The image below shows the repository layout generated by the reference implementation after the first initialization and the addition of a sample target file. The base folder of the repository, which holds the three top level folders (tufclient, tufkeystore, & tufrepo), can be a folder of your choice.
To conclude, TUF is a flexible open source framework that will help you secure your software update delivery system. It does this using roles, and metadata files for each role. The fundamental set of roles, metadata files, and certain parts of the repository layout need to conform to TUF specifications; you also have the option of adding your own set of roles (delegations), and customizing parts (targets) of the repository layout.