Pipelines. An essential part of most workflows, pipelines are the glue that keeps the cogs turning efficiently, correctly and more importantly without severe collapse.
You can usually tell how a person got started in the games industry just by quizzing their knowledge on pipelines. This won’t work every single time, but for the most part, the people you’re trying to decipher will fall into one of these pools:
“What’s a pipeline? / I don’t need one.” “_Insert really complicated answer_” “A pipeline is X to do Y because Z”
The first answer usually comes from developers with less experience in working in a professional/team environment. This is not necessarily a bad thing, mind you — but it either shows their focus was drawn elsewhere or they’re an egotistical airhead that believes they can break years and years of industry research and development into the most efficient workflows. I assume it is usually the former (I hope).
With the second answer, again, there can be a few reasons for a complicated answer. The person could either be really excited at the thought of pipelines, they could be nervous and want to prove themselves or they are relaying information they have researched to you to prove that they understand pipelines. It’s that third sort of person I’m usually wary of.
Whilst it is excellent that the person went out of their way to understand the importance of a strong pipeline, it shows me that it is entirely possible that they haven’t put their theory into practice. Why is this a problem, we’ll talk about it when we get a little deeper.
There is also a fourth possibility to this answer, they could be completely bluffing, but even so, if they can give you an answer that makes sense it at least shows you that they understand the basics.
By now, you’ll probably be able to discern that I personally feel answer #3 is the “correct” answer to give (unless the person is bluffing, of course).
Pipelines, at the core, aren’t that complicated to understand. You can think of pipelines like the flow of the output of team members.
Let’s imagine three developers. One is a great 3D modeller, one is fantastic at rigging and animating 3D meshes and the other is a genius at bringing the output into any project.
Person #A would produce Output #1, which then needs to be passed to Person #B, to be rigged and animated. Once that task has been completed, this output needs to be given to person #C so they can integrate the work into the project.
The pipeline is that journey the output makes to the destination. From Person #A to Person #B to Person #C, or more specifically, in this case, Modeler to Rigger to Engine.
This is an “unoptimized” pipeline and could use a few tweaks but is a pipeline none the less. Almost.
We have to explicitly say what happens to the output during the journey for it to be properly known as a pipeline. Where does it get stored when Person #A is supposed to hand it off to Person #B? How does Person #C get the output to put in the project? Email? Google Drive? USB stick? When you can answer these questions, you’ve got yourself a pipeline outline. Flesh it out and start using it and hey — you’ve got yourself a pipeline.
I know some of you are sitting there right now thinking “What’s the point? Why waste the time, I’ll just stick it wherever and we’ll deal with it when the time comes”. I’m not here to tell you that this is stupid thinking, but I will happily tell you it is heavily misguided.
Before I delve deeper, I’d like to bring up a point that I enjoy reminding people of when I talk to them about their game development plans, especially in regards to new indie studios starting up: These big companies, a lot of what they do costs time and money. The practices they employ usually aren’t for fun and games (see what I did there?) but are instead the product of many, many years of research and development.
As a small developer, you’re in a great position to cherry pick these workflows and methodologies to incorporate into your own business practices without footing the bill for years and years of aggressive R&D. The big cheese has done all the heavy lifting (especially financially) so you don’t have to.
So feel free to check on how their workflows work for them, what parts work and what parts you think are better done elsewhere. Before you know it, you’ll have a rock solid workflow for a fraction of the time and more importantly, cost. But I’m drifting from the original point now, let’s slide back into pipelines.
Once you have figured out where the output needs to go at any point in time, you can start optimising. You know the base time of a lap around the race course, it’s now time to beat your lap times.
This is the point where you usually would start to either test pre-existing tools in your pipeline or would begin R&D into custom tools to optimize your pipeline to be as efficient as virtually possible. This can be anything from a python script that takes the data from A to B or even a custom Maya plugin that automatically rigs and animates the 3D mesh, to save the time of transit and ease the workload of person #B. Time and money saved right there.
So how about a script that takes the output from Person #A that takes his 3D mesh, rigs and animates it then cooks off the data and imports it into the project? That saves even more time and money and allows Person #B and Person #C to focus on more important tasks. Or you could just fire them and save even more money, at that point you have a lot more choices than you did without a pipeline.
It is good practice to employ (or follow) pipelines no matter where you are. At a big company that (surprisingly) doesn’t use a cohesive pipeline? It’s time to talk to your managers. Working with friends on a passion project? Yup, time to get that pipeline running. Even if you’re a solo developer, you should definitely be utilizing a strong and stable pipeline.
There might be a few of you out of there who read this and think “That is nice and all but I don’t really benefit from using a pipeline so there is no point”. I hear this quite often with solo developers and independent developers. It is super important in both of these situations to use a pipeline.
Almost all of the solo and indie developers I talk to end up joining a larger team or a larger company, where a strong understanding of pipelines are required. Truly, the best way to gain a deep understanding of pipelines is to incorporate them into your everyday development, even if you’re a solo developer. That way, when the time comes and you advance deeper into the industry, you’ll have the knowledge and experience to follow and perhaps even evolve any pipelines you come across.
I could go on all day but at this point, I think you get the point. Whether you’re a solo developer or part of a large team, having a cohesive and stable pipeline is essential. A good pipeline says time, money and most importantly — sanity. You like your sanity, right?