Dreamfall Chapters, Extreme Makeover Edition, or: The Rocky Road to Unity 5
For the past four months, Red Thread Games has been working on two concurrent and intimately connected projects. The first is Revelations, the penultimate episode of the Dreamfall Chapters serial adventure. The second is a complete revamp — an extreme makeover — of the previous three episodes.
Yesterday afternoon, the Dreamfall Chapters Unity 5 engine reboot was released on Steam as a free game update.
This launch caps four months of shifting schedules and development delays, a barrage of bugs and a protracted and painful beta phase, a $100,000 investment, and a cautionary tale of the potential pitfalls of episodic game development.
This is the story of how a seemingly simple engine upgrade threatened to derail a studio’s entire future.
Dreamfall Chapters is the episodic sequel to The Longest Journey (1999) and Dreamfall (2006). It’s Red Thread’s first game: we acquired the Dreamfall license from Funcom in 2012, and founded our Oslo-based studio in order to make a sequel. After raising over $1,5 million on Kickstarter, development began in spring 2013. And in October 2014, we released the first of five episodes: Reborn.
For the most part, I believe it was a solid start to the series. The plot took some time to get going, but those who persevered seemed to enjoy the mix of cyberpunk and high fantasy, the original characters and complex dialogue trees, the choice-and-consequence mechanics, along with the — for an indie adventure title, at least — ambitious and beautiful locations.
When it came to performance, however, the game stumbled. It was — this is an understatement — not fantastic. Even with above-recommended hardware specs, many players suffered suboptimal frame-rates and memory-related freezes.
Over the next seven weeks, we pushed out several patches that improved things, but we eventually hit the wall. Unless we went back to split our larger scenes into smaller chunks, reduced visual quality or population density — none of which were ideal — we were done. The performance was not going to improve further.
We needed a miracle.
“I went from nearly 15–25FPS on medium settings in Propast to 47–60FPS the whole time on (virtually) max settings on my i5–2500k and GTX 970!”
“I can finally run the game around 1440p/60fps at max settings, it’s so much smoother and more responsive, a much better experience overall.”
“Finally it’s playable on the highest settings. 60 fps propast, in the worst place 38 fps. Marcuria 50 fps, in the worst place 28fps.”
Subsequent episodes were similarly hampered by the performance bottleneck. While many of the new locations introduced in episodes two and three performed better — a result of lessons learned making Book One— there were only so many graphics we could squeeze out of the engine. We’d run out of graphics. The graphics were spent.
It was in those days that a prophecy, long foretold, came to pass. Its name was Unity 5, and it was the holy grail. It beckoned us, shiny and new. This hallowed thing would not just make things faster, leaner, smoother, allowing for even more graphics, it would improve our workflows, our tools, our lives.
Thus it came to pass that in June 2015, we decided to port the whole game to the new version of Unity.
“The Game looks so much better while running much better than before. Great!!!”
“I played very briefly this morning and it’s looking great!”
“Just downloaded the new Unity upgrade and DFC is looking good!”
When the team returned from summer holidays in early August, we began building episode four using Unity 5, while simultaneously starting the porting process of the previous three episodes.
As I write this, it’s the final week of November. Four months have passed since we sat down to mark our calendars in red ink. The plan was to have the Unity 5 upgrade out in early October, followed by Book Four on October 21st — the anniversary of our first episode.
After all, we’d released three episodes already. We had this shit sorted. We knew exactly how long we’d need to make the fourth one. Our pipelines and processes were tried, true and tested. Our plans were detailed and flawless. The team was motivated, experienced, confident. This was easy. Smooth sailing ahead.
What could possibly go wrong?
The decision to upgrade from Unity 4.6 to 5.2 was based on four core goals:
- To improve development speed, build processes, and game editor performance and stability;
- to increase in-game performance — frame-rate, memory usage, stability;
- to facilitate for forthcoming ports to new platforms, primarily PlayStation 4.
- In addition, Unity had announced they were discontinuing development on the 4.x branch of their engine in order to focus on the 5.x branch.
By early summer 2014, Dreamfall Chapters had grown into a massive game — both in bits and bytes and in time-to-play. In order for our designers and artists to keep working on the sprawling, detailed locations without frequent crashes, we needed Unity new 64-bit editor. The build process was also becoming unmanageable in Unity 4.6. Our build machines were constantly running out of memory and taking hours to complete— if they didn’t simply crash in the process. (They often did.)
Performance improvements was, naturally, a huge motivating factor. The troubles at launch had affected our Steam user ratings and customer satisfaction, and this kept haunting us. We have pride, we care about our games. If there was a chance Unity 5 could improve things, we had to try.
Existing players shouldn’t have to pay for ports to new platforms, but it is something we as a company have to prepare for. The announced-by-Sony-at-Gamescom PlayStation 4 version had been hugely delayed by the lack of support and poor performance in Unity 4. In order to get anywhere with the console version, we had to upgrade.
Finally, it’s rarely a good thing to be left behind by evolving game engines. Unity 4 was being slowly discontinued. Unity 5 was the future. At some point, we’d have to switch, regardless, in order to benefit from further and future enhancements — even from basic bug fixes. This seemed the best possible time to make the switch.
There were enough good reasons to proceed with the port. In August, we did some tests, they went well. Everyone was optimistic. Thumbs up. We expected the port to go smoothly.
(It did not.)
(It never does.)
“I immediately noticed how much better everything looked when I started a new game. They lighting is much more realistic and it adds a lot of depth to the game, and brings out more detail in the textures.”
“I am impressed…the graphics look tons clearer now.”
“Wow! It looks really nice! You can actually tell the difference between the different cloths in Zoë’s outfit now. The jacket looks like a shinier leather.”
Doing episodic development on a story-driven and graphics-heavy game with a team of ten people is a harrowing task. I wouldn’t recommend it to anyone. It’s a relentless and seemingly endless loop of designing, developing, testing, fixing, launching, patching — rinse and repeat.
It’s brutally Sisyphean.
(I’ll probably write about that some other time.)
Combine this with porting 15 hours of scripted (and oftentimes rickety) gameplay to a new engine — one that’s still in development, riddled with bugs and unfinished features — and you have a scenario ripe with drama; a real-life survival RPG.
We’d originally estimated the port to take a couple of months for a couple of developers, working part-time on the project — primarily environment artists, who’d have to go over all the lights in the game to re-bake shadow maps. We figured they could do this on the side while spending most of their time on the next episode. It wouldn’t cost much; it was a no-brainer.
The port ended up taking a team of 4–5 developers four months, spending at least half their time on the port, working overtime to compensate for the mounting delays. It ended up costing Red Thread at least $100,000 in production costs — and that doesn’t factor in the loss of income and added costs to Book Four.
All things considered, the final price tag is probably in excess of $150,000.
That’s a painful amount of money for a small — and cash-strapped — studio like ours.
The reason the port took as long as it did, and why we didn’t anticipate the delays, was a combination of several factors.
There were major changes in the tools — both first-party and third-party — between Unity 4 and 5. Some of the third-party tools hadn’t even been updated, and there was nothing to do but wait and pray for this to happen.
There was an enormous amount of work involved in re-lighting, from the ground up, dozens of enormous locations — more work that we’d initially imagined, more work than anyone could possibly imagine. This was partly due to lacking, and overly optimistic, estimates on our part, and partly due to countless bugs in the new lighting system. In addition, with every lighting fix in the biggest scenes, we had to go through an eight-to-twelve hour baking process before we could see the final results. Rinse and repeat, over and over.
We had to rewrite all shaders and scripts to work with the new engine. And we had to deal with a large number of often crippling bugs, some of which persist to this day, requiring desperate workarounds.
Upgrading from one patch or point release to another would fix one serious issue — say, memory leaks — while introducing a completely different issue — such as broken real-time lights. Every new version of the engine would see improvements in one area, and break the game in two, three, ten other areas.
On some days, it was hard to get out of bed and find the will to keep moving forward.
But we did. We did.
The fact that we did is a testament to the team’s commitment, professionalism and loyalty — and to the simple fact that we had no choice. There was no way back. We couldn’t revert the changes. Book Four was almost finished, and it’d been built from the ground up with Unity 5. Starting over was not an option.
We’d come too far. We had to persevere.
And, yeah, we did.
Some days, I didn’t think we would. Part of me believed this was the end of the road for us, forever stuck in a muddy ditch, hood propped open, engine smoking, as we watched our finances run down and our players vanish in a cloud of dust up ahead.
But despite the bumpy road, Unity 5 has proven to be a boon, a blessing. The new engine has provided us with the tools we needed to not just improve internal production pipelines and optimise performance, but also to up the stakes when it comes to visual quality — particularly in terms of lighting, shaders and materials, and visual effects.
We’ve also effectively future-proofed Dreamfall Chapters — at least for several years — ensuring that we’ll benefit from upcoming bug fixes and 5.x engine updates. The desktop versions will reap the rewards of future improvements, and we’re also in an excellent position to bring the game to new platforms. Like PlayStation 4.
For now, however, performance and prettiness are probably the most immediate benefits.
It’s hard to say exactly how much of a performance increase players will see, but in informal, internal tests — along with several reports from beta testers — the frame-rate has increased by a factor of 50–200%, while simultaneously doubling the distance at which shadows are drawn, and allowing more users to enable the ‘awesome’ setting, along with other visual effects.
Texture materials — such as reflective metals, leather, water, cloth — look more realistic, and the lighting is objectively better across most scenes. In some cases, the differences are minor and the improvements subtle; in others, there’s a literal world of difference.
Several hundred thousands of dollars is a massive pile of cash. Knowing what we know today, the decision to upgrade would have been a lot harder.
We’d still have done it, though.
Fuck it, it was worth it.
The following are screenshots from one of the final beta versions. The release version looks identical or better.
Unity 5 includes a brand new lighting engine called Enlighten. Enlighten has required us to re-light everything. The end result, however, is potentially (and, in many cases, literally) game-changing. The lighting is more subtle, complex, realistic. Built from the ground up using Enlighten, Unity 5 games can look absolutely stunning. And episodes four and five will benefit directly from this.
Materials and shaders — the magic that makes skin look like skin, wood like wood, metal like metal, water like water and shiny surfaces reflective (amongst many other things) — have also been completely switched out and improved, and, in most cases, optimised for performance. Things look better and run smoother.
Visual effects — fire, smoke, fog, clouds — have been replaced with new and (in most cases) improved effects. Wherever we’ve been able to go back and do a proper job of it, things look a thousand times better.
Our artists are still improving character lighting in several scenes (there will be patches). In some cases, like the shot above, backgrounds can look spectacular while characters may appear too dark .
Nothing will look exactly as it did using Unity 4. The lighting model is too radically different for that, and we haven’t attempted to exactly replicate how the game looked before the port. Instead, we’ve strived to create the ambience and look we always wanted to achieve — something that wasn’t always possible with Unity 4, but is now potentially possible with Unity 5 and Enlighten.
Our worlds can, do and will look richer, deeper, tastier after the engine upgrade. There are exceptions, and we’re taking care of those on a case by case basis, but the screenshots — and the game itself — speak volumes.
Book Four: Revelations — our soon-to-be-released fourth episode — has been built from the ground up using Unity 5. The team’s been able to put the new tools, lighting, shaders and effects to excellent use, delivering (in my opinion) the best looking episode so far.
It’s an incredible achievement, and an amazing looking-and-playing experience from a team of just ten people. We can’t wait to share it with the world.
And this is just the beginning of the journey: we will go back and perform more tweaks and improvements as we head into development on Book Five, in addition to the eventual re-release of the complete series — on desktop platforms and consoles.
You can see the Unity 5 version for yourself: the new-and-improved Dreamfall Chapters is available on Steam (and soon GOG and the Humble Store) as a free upgrade for all customers.
(All the images used in this post are available in full-resolution here.)
