Perforce

With Perforce offering unrestricted usage of their revision control server for up to 20 users; it might be time to take a look at what it can offer you as a small independent developer. Even if you, like me, already have been using systems like Git or Subversion to collaborate, it might actually be time to adopt Perforce as your internal revision control system.

This is my atempt at an overview of why a more centralized version control systems has an advantage over distributed systems like Git in some cases and why Perforce has an advantage over systems like Subversion in many of those cases. It also serves as an explanation to why Perforce adoption has become so widespread in large parts of the game industry.

Coming from a CVS and later Subversion background it took me a while before I finally “got” Git. But once you “git it” you feel like a free soul, a drifter, a programmer who can code wherever he want, whenever he want. All you will ever need is in your digital backpack of a hard drive. An invincible ninja who can sneak in, branch, fix and merge stuff however he wants. Git is indeed an incredible programming tool that let you code fearless.

However, Git doesn’t quite suite every workflow. Because it was developed to suite one well defined set of requirements, and does it very good, it will have problem to cope in other areas that it wasn’t designed for.

Let me start the explanation with the problem of binaries.

In Gits world, and mostly traditional version control wisdom, you should not really check in any binaries. As these are generally generated, only the source files and build system necessary to generate them should be checked in. One good reason for this is that unlike source files with distinct lines and meaning that can be interpreted by a human, binary files are difficult to delta in the first place and it is often impossible to merge and sort out any conflicts between two binary files in a meaningful way. You simply have to choose between them and accept one of the two binaries as the current version.

Another problem is their file size. They tend to be quite sizable. As many binaries are hard to delta in the first place, you may end up storing several copies of the full file, inflating the repository as well. This makes one of Gits many advantages a major drawback when repositories contain a lot of binary files. To be able to be fully distributed, were all repositories are equal, Git need to download the whole repository with all history and deltas when you clone it. This means that if the whole repository, not just the latest version of the files, is a gigabyte of data, everyone who work with that repository need to download and keep their own copy of that gigabyte of data.

So the solution in decentralized system is either to live with it or to simply not do any revision control of binaries. Which is more akin to two non-solutions as they sooner or later will break down under the crushing weight of data.

This might just be a minor annoyance if all your projects mainly only consist of source code, but not if your occupation is to write games where more than half of your source files are art resources in various binary file formats. This is why we need to turn our attention to another solution.

A less distributed solutions, like Perforce and Subversion, with a central repository on a server only need to download the data so it can update to the version of the files that you requested. Which makes it more ideally suited for repositories that contain a huge amount of binary files.

In Subversion as well as Perforce you can organize your files however you want. While Subversion creates a trunk, branch and tag directory when a repository is initialized, it isn’t necessary to follow that template. You can just as in Perforce organize the repository however you want, just as you would normally do in the file system of your own hard drive. Also, you do not have to download the entire repository tree in neither system, just a subset of it. If you used the standard Subversion template you normally just download the trunk directory or a part of it. The same applies to Perforce, although there is no standard template directory layout.

The first big difference between the two systems is that Subversion record the state of the whole repository for each revision. It is the repository as whole that is version controlled from the users perspective. While Perforce record the state of each file individually, each file has their own revision, but group them into changelists that fill a similar role as Subversions “revisions” or Gits concept of “commits”.

This makes the second big difference and Perforce most powerful feature possible: The possibility to mix and match file versions however you want in a checkout, they do not need all to be the latest versions, as well as checking out whatever subset of the repository and organizing it locally however you want.

In Perforce a repository is referred to as a depot, which is where the files are organized on the server. The client side on your computer, where the files are downloaded to be worked on locally, is referred to as a workspace. What files are downloaded from the depot to the workspace, what version of them, where they are placed in the workspace and how they are named is all controlled by the current workspace view. Every user can have their own custom workspace view if they so wish. Perforce keeps track of how files in each workspace map to the files in the depot.

This makes it possible to put even more things under the control of Perforce without running the risk of bloating anyone’s workspace. In a heterogeneous project, were not everyone is expected to be able to generate all files, artists are not supposed to build the software and programmers are not supposed to export game content into runtime formats, it makes a lot of sense to put all the generated files (possibly excluding many of the intermediate ones) under revision control, including builds of the software.

It also makes the organization of the depot less of an issue. You can easily keep the directories containing builds in a separate directory structure from the source tree, in addition to keeping shared engine builds and source trees separate from individual game assets and source trees.

Another feature of Perforce is that all the files in your workspace are read only by default. You have to tell the system that you want to edit a file. This can at first seem like a step back to a non-concurrent way of working. However, this is just to inform your colleagues of what files you are editing and making explicitly stating that you intend to edit something a standard operation in the system. You and your colleagues can still work on files and edit them concurrently, although you have to merge to resolve any conflict upon submitting the changes.

So what is the upside of having that as a standard operation? Because it makes the locking of files a natural extension of the system and not an edge case. In addition to telling the system that you intend to edit the file you also have the option to ask the system to lock a file for edits by others which prevents concurrent edits and solves the issue of binary files that can’t be merged in a meaningful way in the case of conflicting edits. Which is perfect for artists.

While you can manage this workflow by manually using the Perforce CLI and GUI client, this way of working really calls for good integration with IDEs and authoring tools to make things go smoothly.

But does that mean you have to give up on Git in favor of Perforce? Absolutely not! Git is still an amazing tool to develop code with. Perforce offers Git-Fusion if you want to go the whole mile for total integration or you can just use Git-P4, if all you want is just to “clone” parts of your depot to temporarily work with Git before submitting back the changes to the depot, a Git command that provides a way to interact with Perforce depots.

Anyhow, this was a brief overview of why more centralized systems has an advantage over distributed in some cases and why Perforce has an advantage over Subversion in many of those cases.

In my next post I’m going to show you how I work with Perforce to manage a real project that takes advantage of the awesome stream depots that pull together several of Perforce features in a coherent way and simplify the management.

Originally published at en.dahnielson.com on July 28, 2014.