How to publish artifacts from Bazel to Maven Central

Natan Silnitsky
Apr 26 · 7 min read

Recently, I managed to deploy several OSS artifacts built with Bazel to a Maven Central repository. As this process is quite complicated and involves many steps, I’ve decided to write a detailed step-by-step guide. I sure would have liked to have had one before I got started.

Image for post
Image for post

I work at Wix, as part of the data-streams team, that is in charge of all our Kafka infrastructure. Recently we started to open source our tools and libraries, starting with Greyhound — a Scala/Java High-level rich SDK for Apache Kafka.

Greyhound is built with Bazel — an open source build tool designed for speed and accuracy with truly incremental builds and a high degree of test parallelism.

In order to publish Greyhound’s logical artifacts to maven central using Bazel, we needed to perform some preparations.

First, Bazel does not publish build output jars in a local/remote maven repository by default. A separate publishing ruleset is needed. Thankfully, graknlabs has created and open-sourced such a ruleset.

Second, with Bazel it is encouraged to have smaller build unit targets at java package level — it doesn’t make sense to publish each individual package as a complete module to Maven central, such that an additional target that aggregates all the relevant packages has to be created for each logical maven module.

The guide below provides all required steps to deploy to Maven central including all Bazel related setup.

Step 1: Create a Sonatype JIRA Ticket

First you need to create a Sonatype JIRA account. (Sonatype is the organization that manages Maven Central these days). Keep your credentials somewhere safe, you are going to need them later on.

Once you are logged in, create a new issue.

You need to do this in order to allow your Sonatype user credentials to publish your OSS project’s Maven modules to Maven central.

Image for post
Image for post
Sonatype JIRA ticket

You need to specify group id, project url on github, etc…

It will take some time for the administrator to get back to you — that’s why this is step 1.

Step 2: create and distribute GPG key

In order to be able to successfully publish your artifacts, they are required to be signed using GPG keys.

You need to make sure to create a set of private/public keypair on the machine you are going to publish the artifacts from.

On a Mac, the easiest way is to install GPG Keychain, and follow the instructions on how to create a new keypair.

Image for post
Image for post

Alternatively, for linux machines, you can use GnuPG.

Don’t forget to also publish the public key to a key server, so that your signed artifacts would be verifiable.

You can verify that the keypair is available by runninggpg --list-secret-keys

Step 3: Setup Bazel configuration

1. add a dependency on bazel-distribution repo from graknlabs

2. for each artifact you want to publish to maven central, create an aggregate bazel target that includes sources for all finer-grained bazel targets (if you have bazel targets with coarse-grained granularity you can skip this step)

aggregate bazel target will all content of Maven artifact

A few notes:

  • This is a scala_library, but this can work for java_library as well
  • All the labels in srcs are actually filegroup with appropriate visibility settings that allow out-of-package dependency:
  • This target is tagged as "manual", so it won’t be needlessly built when running bazel build //core/… as any source change would cause the whole source set to be rebuilt for this specific target.
  • There is a special maven_coordinates tag that helps the assemble_maven rule to fill-in details in the pom file.
  • You also have to add a maven_coordinates tag to the 3rd party dependencies (such as dev_zio_zio_2_12 in the deps attribute of the example above) so that they will appear as dependencies in the pom file:

3. Add assemble_maven target for each artifact you want to publish. It will create all the necessary files for your artifact, including main jar, source jar and pom file. Enter all project details for the pom file.

Notes:

  • For the target attribute you should put the label for the Bazeltarget (scala_library/java_library) you created in the previous step with all the relevant sources.
  • Make sure the project_name and project_description are unique for each of these targets/artifacts
  • The VERSION file just contains the SEMVER, e.g. 1.0.0
  • choose from either apache or mit licenses

4. Add deploy_maven for each artifact you want to publish.

The deployment.properties file should contain the url for the sonatype stating area:

repo.maven.release=https://oss.sonatype.org/service/local/staging/deploy/maven2/

For a complete working solution you can review our greyhound repo.

5. Build the assemble_maventarget and review the generated pom file and jar file in the bazel-bin directory. Change the targets configuration as needed.

Step 4: publish via Bazel to Sonatype staging area

Once you get the ok from the sonatype administrator in the JIRA ticket you opened in step 1, you can try and publish the artifacts to the Sonatype staging area.

You need to have your Sonatype account credentials (that you created in step 1) available as environment variables for successful execution of the deploy-maventarget:

make sure that you escape any special characters found in the password.

For each artifact, execute:
bazel run //path/to/artifact:deploy-maven -- release --gpg

  • Make sure that you didn’t skip step 2 and the GPG keypair is available on your machine by runninggpg --list-secret-keys
  • Make sure you put the version you want in the VERSION file that depends on

If the run is successful you will find the artifacts in https://oss.sonatype.org/#stagingRepositories

Check the checkbox and in the bottom part, go to the content part and review the artifacts content.

Image for post
Image for post
comwix-1665 staging repository content

You can click on each item and review its content. I recommend that you review the pom file content, download the jars and verify their content using jar tf path/to/jar/file.jar.

Step 5: Auto-evaluation of the published artifacts content

When you are satisfied with the content, the next step is for Sonatype to automatically verify that the content is valid.
Click on Close repo.

Image for post
Image for post
Close button for staging repo

The evaluation process will start. This can take a few minutes.

It includes verifying that the pom file is in order and that the GPG signature is valid.

You can review the evaluation results in the Activity tab.

Image for post
Image for post
Staging repo Activity tab

When the evaluation is completed successfully, the repository will close.

Step 6: Release the closed staged repo to Central

Click on Release in order to start the sync process of the artifacts to Maven central itself.

Image for post
Image for post

This can take several hours, up to a day.

Once the sync is completed, you will be able to find the artifacts in https://search.maven.org

Image for post
Image for post

Congratulations! You have successfully published your artifacts from Bazel to Maven Central.

I would like to thank Valery Frolov, Ittai Zeidman and Dmitry Karlinsky for their help in getting this process to work for our Greyhound project.

Here is a link to Sonatype’s oss publishing guide.

Here are links to my blog series on how to successfully migrate to Bazel from Maven or Gradle:

Also a link to an article about Exodus — our automatic migrator tool from Maven to Bazel.

Thank you for reading!
If you’d like to get updates on my experiences with Bazel, Kafka and OSS projects please follow me on Twitter and Medium.

You can also visit my website, where you will find my previous blog posts, talks I gave in conferences and open-source projects I’m involved with.

If anything is unclear or you want to point out something, please comment down below.

Wix Engineering

Architecture, scaling, mobile and web development…

Natan Silnitsky

Written by

Backend Infrastructure Developer @Wix.com

Wix Engineering

Architecture, scaling, mobile and web development, management and more, written by our very own Wix engineers. https://www.wix.engineering/

Natan Silnitsky

Written by

Backend Infrastructure Developer @Wix.com

Wix Engineering

Architecture, scaling, mobile and web development, management and more, written by our very own Wix engineers. https://www.wix.engineering/

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store