Publishing artifacts with AWS Codeartifact and GitHub Packages

Photo by https://unsplash.com/@curology

Exploration is and always was part of the Reach-Now tech DNA. While we have an established way to publish code artifacts to an artifacts store, we are keen to learn more.

We picked

• AWS Codeartifact
• GitHub Packages

because both AWS and GitHub are used at reach-now tech for years. While AWS Codeartifact was announced earlier this summer, GitHub Packages are generally available since November 2019.

Reach-Now’s main tech stacks are Typescript and Kotlin. Our exploration published Kotlin/Gradle Jar files as well as NPM packages files. The exploration code is open source.

Publication Table

+------------------------+--------------------+--------------------+
|                        |  AWS Codeartifact  |   GitHub Packages  |
+------------------------+--------------------+--------------------+
| How to publish         | CI/AWS CodePipeline| GitHub Actions     |
| How to authenticate    | Temporary Token    | GitHub Access Token|
| How to fetch artifacts | npm scope for the  | via package manager|
|                        | organization;      | (gradle, npm)      |
|                        | repository entry   | facilities after.  |
|                        | in build config    | package manager    |                                
|                        |                    | login with packages|
+------------------------+--------------------+--------------------+

AWS Codeartifact

AWS CodeArtifact, a new addition to AWS’ vast landscape of services, provides a (private) package registry for various language ecosystems. Among those are Maven and NPM repositories, which we evaluated. The code examples and instructions can be found here.

Repository Setup

As of today, there is no CloudFormation Resource to instrument a CodeArtifact Repository, so you’d have to resort to Terraform or create the resources manually:

# create domain
aws codeartifact create-domain --domain some-domain
# create repo
aws codeartifact create-repository \
  --domain some-domain \
  --repository some-repository

(A Domain is some kind of namespacing primitive in CodeArtifact.)

Access Tokens

Access to CodeArtifact is granted via temporary tokens, which are created upon request by an authorized user. This is a good practice, since tokens often leak into public repositories, Docker images, console output, etc. and the security implications of someone having write access to your private repositories are pretty grim. By default the token is valid for 12 hours.

export CODEARTIFACT_AUTH_TOKEN=$(aws codeartifact get-authorization-token \
  --domain some-domain \
  --domain some-repository \
  --query authorizationToken \
  --output text)

NPM

In a library (e.g. some-library), you want to publish, create a .npmrc file for your namespace/organisation (e.g. @reach-now). We’ll resolve & use the endpoint of the repository we created above:

cd npm/some-library
export REPOSITORY_ENDPOINT=$(aws codeartifact get-repository-endpoint \
  --domain some-domain \
  --repository some-repository \
  --format npm \
  --query repositoryEndpoint \
  --output text)
cat << EOF > .npmrc
@reach-now:registry=$REPOSITORY_ENDPOINT
${REPOSITORY_ENDPOINT#https:}:always-auth=true
${REPOSITORY_ENDPOINT#https:}:_authToken=\${CODEARTIFACT_AUTH_TOKEN}
EOF

Modules will be pushed and pulled to the repo when using this prefix:

npm install
npm publish

In an application, which is importing some-library, we need to have the same npmrc file. We then should be able to install and use the library:

cd npm/some-app
cp ../some-library/.npmrc .
grep some-library < package.json
	"@reach-now/some-library": "^1.0.0"
npm install
node index.js
ohai: 🔔 🔔 🔔

Gradle

In a library (e.g. some-library), you want to publish, create a gradle.properties file, which includes a reference to the Repository’s endpoint:

export REPOSITORY_ENDPOINT=$(aws codeartifact get-repository-endpoint \
  --domain some-domain \
  --repository some-repositiory \
  --format maven \
  --query repositoryEndpoint \
  --output text)
echo "reachnowRepoUrl=$REPOSITORY_ENDPOINT" > gradle.properties

In build.gradle.kts (we’re using the Kotlin DSL in this case) we add the maven repository as a publishing target:

publishing {
    ...
    repositories {
        maven {
            val reachnowRepoUrl: String by project
            url = uri(reachnowRepoUrl)
            credentials {
              username = "aws"
              password = System.getenv("CODEARTIFACT_AUTH_TOKEN")
            }
        }
    }
}

After this step we should be able to publish the library to the repository:

gradle clean build publish

In an application, which is consuming the above library, we create a similar gradle.properties file containing the repository name:

cd gradle/some-app
cp ../some-library/gradle.properties .

In our projects build.gradle (groovy syntax this time) we reference the repository, so we can fetch the dependency:

repositories {
    mavenCentral()
    maven { url "https://jcenter.bintray.com" }
    maven {
        url reachnowRepoUrl
        credentials {
          username "aws"
          password System.env.CODEARTIFACT_AUTH_TOKEN
        }
    }
}
...
dependencies {
    ...
    implementation "com.reachnow:some-library:0.0.1"
}

After building & starting our application, we can query an endpoint using our library’s code:

gradle clean build jar
java -jar build/libs/some-app-0.1-all.jar

curl localhost:8080/hello
Bus

GitHub Packages

We use private GitHub repositories for all kinds of software development quite extensively. Github.com/reach-now/codeartifact-packages-publishing/packages explored the publishing flow for private repositories. It is now open source to share it with the community. While deleting packages within private repositories is possible at any time, deleting packages within public repositories is restricted.

You can publish quite some formats to packages, however in case your favorite format is not supported, you can package as Docker images and publish those. In this case Docker image tags can be customized.

The preferred way to publish to packages is GitHub Actions. A GitHub event triggers an Action workflow. In case your trigger is different, you can also create outside events.

Publishing the built artifacts is two fold:

• adapt the package manager descriptor file
• configure the token within Action workflow files

Creating and managing tokens for Action workflows is described in the variables & secrets in workflows section.

Gradle

We leverage Gradle’s Maven Publish Plugin within the Gradle Kotlin DSL script to publish:

publishing {
    repositories {
        maven {
            name = "GitHubPackages"
            url = uri("https://maven.pkg.github.com/reach-now/codeartifact-packages-publishing/")
            credentials {
                username = project.findProperty("gpr.user") as String?
                password = project.findProperty("gpr.key") as String?
            }
        }
    }
    publications {
        create<MavenPublication>("gpr") {
            from(components["java"])
            groupId = groupId
            version = version

        }
    }

source

Also we need to provide authentication details within the action workflow file:

- run: ./gradlew \
           -Pgpr.user=[username]  \
           -Pgpr.key=${{ secrets.[GitHub-secret-id] }} publish

source

In an application we set up gradle properties to store credentials that allows this repository’s code to fetch jar artifacts from private GitHub repository packages.

mavenUser=[Your Github Username]
mavenPassword=[GitHub Personal Access Token]

Install the dependency

./gradlew install

and run the application

./gradlew run

Check the application response in another terminal

curl http://0.0.0.0:8080/lib
Kotlin

NPM

We leverage package.json as one option to configure npm. The target repository to publish would be:

"repository": {
    "type": "git",
    "url": "ssh://git@github.com/reach-now/codeartifact-packages-publishing.git",
    "directory": "packages/name"
  }

source

Providing the authentication details is done within the action workflow file:

- run: npm publish
  env:
    NODE_AUTH_TOKEN: ${{ secrets.[GitHub-secret-id] }}

source

In an application we install the package from GitHub package manager scope (see also Authenticating to GitHub Packages):

$ npm login --registry=https://npm.pkg.github.com
> Username: USERNAME
> Password: GITHUB_PERSONAL_ACCESS_TOKEN_VALUE
> Email: PUBLIC-EMAIL-ADDRESS
$ npm i
$ node index.js
ohai: 🔔 🔔 🔔

Conclusion

Both, AWS Codeartifact and GitHub Packages are ready to host your packages and can be integrated into our workflow. We are technically prepared in case we get the task to change our artifact management setup after showcasing artifact publication with both artifact stores.

You can find the main steps of the exploration also in a gist:

Which package managers are you going to explore next?

Magnus & Lothar