Run your own Unity Package Server!

🆕 Find an updated version of this article on my new blog: https://markus.hofer.rocks/package-server-google-cloud

ℹ️ If you want to run your package server on a QNAP NAS instead, check out this article: https://markus.hofer.rocks/package-server-nas

A little while ago Unity introduced its Package Manager, a nice and simple way to bring reusable packages into a project while keeping track of dependencies and compatible Unity versions.

Looking at the Package Manger package in the Package Manager

How great would it be if we could use this system for our own reusable assets, if we could store them on our own server and have them show up in the package manager too?

Well, as it turns out we now can! (Unity 2018.3+)

Unity is using a normal npm package server to serve their packages. We can simply set up our own npm package server, hook it into the project manifest and have the package manager serve our own packages in addition to the ones from Unity!

Unity is using Artifactory, but we’re going to install Verdaccio (a lightweight and free npm registry server)! We’ll put it on a Micro Instance on the Google Cloud Compute Engine (but you could just as well have it run on your local PC).

Let me walk you through it…

Step 1: Creating a server instance

Go to your Google Cloud Platform Console, choose Compute Engine > VM Instances and click on CREATE INSTANCE.

Give it a name, choose a region, select micro as the machine type and click Create to create your Debian 9 Linux Server with 10GB of storage.

Notice how it estimates the monthly cost as $4.28, but it could be free for you if you chose one of the regions that fall into google’s always free usage limits (They give you one f1-micro instance in us-west1/central1/east1 with 30GB storage for free each month)

With our instance created and running, we’ll now connect to it by clicking on SSH.

Connect to your new instance

This will open up a linux terminal in a browser window

This is how we talk to our debian linux server

Step 2: Installing and Configuring Verdaccio

Prerequisites

Now we’ll install or own package server, Verdaccio. We want to do this as the root user, so we’re going to create a password for the root:

$ sudo passwd

Think of a password and enter it. And enter it again.

Now log in as the root (su = superuser = root):

$ su

It asks for the password you just set, enter it and you should now be logged in as the root user!

Next we’ll install NodeJS and npm (the node package manager)

$ curl -sL https://deb.nodesource.com/setup_11.x | bash -
$ apt-get install -y nodejs

nodejs and npm are installed!

For both these steps you’ll see it installing things, no errors should come up.

To verify that both nodejs and npm installed correctly you can check their versions like this:

$ nodejs -v
$ npm -v

If each command shows you a version, then it worked!

Installing Verdaccio

Now we’re ready to install Verdaccio!

$ npm install --global verdaccio

Let it install. Once it’s done switch back out of super-user mode by either pressing CTRL+D or typing exit, because while we wanted to install it as the root user, we want to run it as a normal user.

We’ll configure it in a second, but first we need to run it to make sure it generates its config file.

$ verdaccio

Then press CTRL+C to kill it and we’re ready to change the config file!

$ cd verdaccio
$ vim config.yaml

This opens the config.yaml file in the VIM text editor. No need to panic, I’ll walk you through it.

All we need to do is add two lines to tell verdaccio to listen to all signals from everywhere as long as they come through port 4873:

listen:
-0.0.0.0:4873

Press i for insert. Add the two lines, then press ESC to come back to command mode.
Now type :x and press Enter to save the file and quit VIM.

Let’s run verdaccio!

$ verdaccio

Verdaccio is running on port 4873!

Firewall

Great, it’s running! But we need to tell the firewall around our server instance to un-block port 4873 so we can communicate with Verdaccio from the outside!

Go to the Google Cloud Platform Console > VPC network > Firewall rules and click Create Firewall Rule.

Add a Rule to the Firewall

We want to create a rule that allows incoming traffic (Ingress) from all possible IPs (0.0.0.0/0) for tcp port 4873.

I marked the important bits in yellow. Turns out most of the bits were important…

And we’re done installing! Verdaccio is running and accessible from the outside world! Point your web browser at http://[your server’s IP]:4873 and you should see this:

Verdaccio is running, but empty

Keeping Verdaccio alive

Nice! The only problem is: As soon as we close the console window, Verdaccio gets shut down, so we have to work around that with a little tool called screen! (should be installed, if it’s not you can get it with sudo apt-get install screen)

Press CTRL+C to kill Verdaccio and type

$ screen

This opens up a new virtual screen, a kind of safe place, which is where we will now launch Verdaccio!

$ verdaccio

Then we simply press CTRL+A and CTRL+D to detach the screen session.

It’s still running in the background with Verdaccio safe and sound inside it and we can even get back to it at any time by typing screen -r, but most importantly we can now close our terminal window without killing Verdaccio!

Now we’re ready to create users, add packages, and hook it up into Unity!

Step 3: Creating a user

Let’s open PowerShell (or something similar) on our local machine and create a user! (You’ll need npm for this — if it’s not installed you can get the installer here — remember how to check if it’s installed? Works here too!)

> npm adduser --registry http://[Server IP]:4873

Enter a username, password and email address and you should be created and logged in!

User created!

Later on you can log in with:

npm login --registry http://[Server IP]:4873

Step 4: Creating and uploading packages

The process for this is simple: Put all the files into a folder, add a package.json file that holds all the relevant information and upload everything via npm!

I recommend having that folder in a Unity project, because the resulting package will not work if a meta file is missing and having it inside Unity makes it easy for you to see if everything is in order.

Let’s add a package.json file to the root of the folder that looks like this:

package.json for a package without dependencies

Note how you can define a required unity version, description and other things. You could even add a README.md file.

You also need to have assembly definition files in there or it will not compile (as the actual files will reside outside the Assets folder), so make sure those are set up correctly! (I wrote a little post about how to deal with them too.)

Your final folder could look like this

Once it’s all set up, got to the folder you want to publish in PowerShell and do:

> npm publish --registry http://[Server-IP]:4873

npm and Verdaccio will do the rest and your package will now be visible on Verdaccio’s web interface!

Updates

If you want to release an update to your package, simply increase the version in your package.json file and publish the same way!
Verdaccio will keep the entire history of your packages so dependencies will always bring in a compatible version.

Dependencies

If your package depends on other packages, simply add those packages’ names plus their required versions to the list of dependencies in the package manifest:

package.json for a package with dependencies

Now whenever you install this package the Package Manager will make sure the dependencies are there as well (they won’t show up in the Packages folder in Unity, but they’ll be there)

Step 5: PackageManager, meet Verdaccio!

Now there’s only one last thing to do: Get the Package Manager to talk to Verdaccio and show our custom packages!

Open Packages/manifest.json in your Project folder and add a scopedRegistry with the url of your server:

Add this to the top of your Packages/manifest.json

This is enough for packages you publish to your own server to show up in Unity’s PackageManager, as long as their names are within the given scope.

So if your scope is com.company.tools, then com.company.tools.mytool will show up, but com.company.mytool would not show.

The Package Manger will search all registries for packages that are within the given scopes. You can add multiple scoped registries with multiple scopes each. You can even have the same package names on multiple registries and it will pick the one from the registry with the more specific scope.

Our own package in the Unity Package Manager, served from our own npm package server!

Done! Awesome!

Bonus

I have to say that I’m pretty pleased with our progress at this point, but of course there’s always something that can be improved upon…

Unpublish (remove) a package from the server

You can get rid of packages too. This will remove all versions and get rid of the package entirely:

npm unpublish --force at.blackish.testpackage --registry http://125.138.25.12:4873

Restrict access

At the moment everyone who knows the IP address of our server can create users, get packages, add packages and even delete packages. That’s not exactly ideal. I should probably look into that…

Just kidding, I did look into that. The simplest way to at least stop everyone from uploading and deleting packages is to restrict registration. Once all your users have registered you can simply set max_users: -1 in the config file and no one can register anymore.

All your user data is stored in the htpasswd file, so you could even set up all users by hand (tools like this one can help you md5-hash your passwords) and having registration disabled from the start.

Reserve a static IP address

The way we have set it up the external IP address of our server instance changes every time we stop and start the server. We don’t plan to do this very often, but it might still be a good idea to go to VPC network > External IP addresses and switch the type of our IP address from Ephemeral to Static.

Google is only charging for reserved IP addresses that are not in use, so if you have your server running continuously, it should not add any extra cost. But if it’s not in use the reserved IP will cost $0.010 per hour — almost twice as much as the $0.006 per hour it costs to run our micro instance. Keep that in mind…

Remove Uplinks

Uplinks are a way to connect Verdaccio to other npm registries. If a package is not found on your server it would then look in those other places and potentially cache it locally. This feature has potential, but I’m pretty sure we don’t want our server to look for unity packages on the npmjs registry, so let’s comment that out for now:

Little use looking for unity packages on the npmjs registry