Grand Theft Quantum
One artist’s journey to create a GTA-style city environment in Unreal Engine
Hello, Tav Shande here, today I’m super excited to share the making of my latest project, Grand Theft Quantum. Back in late 2016, I shared the making of my Unreal Engine Virtual Luxury Camping scene and since then I’ve been working on something special in my free time. But first, my team of lawyers won’t stop fidgeting until I share the following disclaimer: Opinions expressed are solely my own and do not express the views or opinions of my employer. Oh wow, that worked like a charm! They look much happier now. Okay, let’s dive right in.
My art career started in 2003 using Unreal Engine 2.5 on the ‘America’s Army’ PC game and since then I’ve worked with Electronic Arts, Crytek, and Activision using game engines such as Renderware, CryEngine, & Radiant. Each engine has its strengths and weaknesses but UE4’s ease of use and art-friendly toolset allows me to create anything I can imagine. I keep coming back to Unreal for all my personal 3d environment projects because I feel creatively empowered while using it and in total awe of the robust game engine the team at Epic Games has created. The Engine was easily my first choice when deciding how I would tackle the ambitious goal of re-creating a massively detailed one-by-one kilometer section of downtown San Francisco. If you want to learn more about how I decided to approach this project, and why, plus some lessons learned while doing so, you're in the right place.
Why ask why?
Asking yourself ‘why’ can be a powerful question. In this case, the question I kept asking myself was, “ Why aren’t more 3D artists creating large-scale, highly detailed, realistic, urban environments that look good from street level & from the sky?” All the 3D scanned cities I‘ve seen looked great from 30,000 feet in the sky but fell apart into a lumpy mess with numerous lighting, geometry, and texture artifacts at street level. Many of the purely procedural cities I had seen didn’t actually look like real-world architecture. During my pre-production quest to find good examples of digital real-time cities I made a mood board of high-quality, real-time digital cities and noticed the best looking ones from street level were usually heavily hand-crafted.
Over time, as procedural game pipelines improved, I noticed that the more convincing real-time city environments came from development teams who not only cared about street-level accurate visuals but also leveraged procedural systems to provide the necessary scale needed to depict whole cities. A time-consuming component of creating believable procedural city systems is deciphering the complex decisions an artist’s brain would make while art directing and crafting any given asset. Crafting a purely procedural city quickly becomes an attempt to code enough rules that mimic those thoughts while trying to hide any patterns introduced by lines of coded instructions unsuccessfully imitating the mathematical complexities of chaotic dynamical systems. Ideally, a team would want procedural tools to maximize an artist’s impact and output while never taking away from the visual accuracy and art direction of the scene being crafted. It’s an interesting and extremely tough problem to solve, however, it wasn’t the goal of this project. This project was about challenging myself to make an art-driven, unique, hand-crafted, large-scale bustling city.
Some examples of large real-time cities I really admire are GTA 5, The Division, Spiderman, & the recently released UE5 Matrix experience by Epic on consoles which is hands down the best tech demo of a massive, highly detailed, real-time city I’ve ever seen to date. So why aren't more artists creating environments like this? I wanted to find out the answer but first I needed an interesting goal to build towards. This is where my love for films and open-world urban sandbox games came into play. I started writing the story for my very own city-based GTA-style trailer, Grand Theft Quantum which would ultimately help me finish this ‘epic’ project.
Grand Theft Quantum: Ideas, inspiration, & The North Star
Not knowing exactly how I was going to achieve the task of creating a massive city excited me. I knew it would take me beyond my comfort zone due to the scale of the task and that’s where growth happens. In addition, I would have the opportunity to try out many new tools and techniques. Beyond just starting a large 3D modeling/texturing exercise, I wanted to find my ‘North Star,’ a guiding light that would keep me going if I ever felt like giving up. I needed to tell a story with this piece. Storytelling is a powerful method for learning and motivation, it’s a fundamental part of being human and art is storytelling. I quickly realized that if I wanted to finish this multi-year project I would need a compelling backstory for the world I was about to create. I grabbed my sketchbook, briskly turned past my drawings & random rap lyrics to a blank page, then started writing.
No Time for a story about Time?
The idea that kept me excited for the duration of this project was the Sci-Fi story I had crafted in which the digital city of SF would serve as an appropriate silicon valley themed backdrop. The title: Grand Theft Quantum is in reference to a fictional futuristic law that passes making time theft a felony. In this, not so distant future the main character, a nameless, mysterious, Afrofuturistic female clad in all black, is a Time-Broker who suddenly re-appears in the Bay Area after a large quantum computing tech firm first discovers how to manufacture time crystals. The story starts out in a police station interrogation room right after one of the Quantum detectives asks the broker why she’s back in San Francisco. Originally her answer to that question would be the voice-over to the trailer.
This mood was heavily inspired by and also a nod to one of my all-time favorite games series, GTA, specifically, the masterful Los Santos trailer. This Cowboy Beepob animated movie intro was also a huge inspiration. In the end, the idea would require too much time creating characters and animations for scenes that didn’t really serve my final goal of building a detailed city so I put those story elements on hold for the trailer and stuck with the plan of showing a digital slice of life in the Bay Area around downtown SF. Which could also serve as B-roll If I ever decided to make the character-driven trailer.
Plans can evolve and that’s okay
Originally I chose to only re-create Howard Street and the buildings on both sides but that quickly expanded into creating a one-kilometer square scene. It didn’t take much time after starting construction to realize in order for this digital city to feel like its real-world locale, I needed to replicate the long sightlines and connected road networks, otherwise, it could feel artificial like an enclosed movie set.
When I realized I was now signing myself up to do 36 blocks of downtown SF instead of only one street, two things kept playing back in my mind. The first thing was me laughing to myself doubting I would ever finish, telling myself in my best Jay-Z impression, “ You crazy for this one…” The second was me asking myself how I felt about outsourcing. After settling on a good path forward it was time to start building.
Building the city
Don’t worry, this section won’t be overly technical. I’ll simply provide a high-level overview of my process and dive deeper into specific methods that may be interesting to someone who might want to build something similar. The common methodology for creating realistic assets in 3D remains the same for me, all the small details make up the larger picture and If it’s on the real object that’s being made then it should be represented in the final digital model. With that in mind, any asset can be placed in the scene and it will feel like it belongs to the same world, with the same level of detail. Before jumping into the smaller details it’s always good to block out all the large general shapes for a unified world scale.
Materials
Materials define how light interacts with the surface of your mesh so it’s one of the most important things to get right when aiming for realism. Making sure the roughness map, and other texture map values are physically correct can really tie a whole scene together. Materials plus accurate lighting and shadows are really the holy grail of CG rendering. For this project, I used one of my favorite pieces of software, Substance Painter. For the Bay Bridge, which was about a mile long, I broke it into modular & mirrorable parts then created PBR textures for each segment. The bridge has four materials, each part has a high-resolution mesh transfer. I then exported the PBR channel-packed textures to be imported into Unreal and applied them to the bridge static mesh. I split the mesh into smaller parts for LOD’s and made it a blueprint.
Roads
I started by working on the large elements in the space. The first thing I did was establish the road by making a large top-down reference texture map of the area in Maya. With a 2d map of the area assigned to a plane, then I started modeling the road. For simplicity, I kept the world flat but matched the bowed segments of road edges that dip down towards the gutters. After the modeling was completed I added UV maps to the road mesh so all segments were facing the same orientation, allowing the road texture to flow in the same direction around any curves in the geometry. To hide any seams in the road texture which would naturally occur at the UV borders, I created a layered material that allowed me to paint perfectly tiling road textures across any seam, hiding it. The road has about 4 layers all with varying degrees of dirt, damage, cracks, water, and grime. Finally, it’s good practice to split the road into optimal smaller sections so it can be occluded and the whole 3D mesh isn’t always drawing in the world.
Sidewalks
After the roads, I created the sidewalks. I made a modular sidewalk set that consisted of right-angle segments, driveways, straight and corner parts with ramps, etc… I brought these into the editor and used them with the Snappy Roads blueprint I purchased on the UE4 marketplace then manually started creating my sidewalk sections over the map guide texture. Once all my sections were in their final positions, I merged the meshes to create new static meshes. I kept the spline sidewalk pieces on a separate layer in case I needed them later. After completing each block I then filled in the central area of the block in Maya as a foundation for where all the buildings would sit.
Buildings
For creating buildings a trick I use to get proper scale, besides measuring everything against real-world objects, is to make sure you create the door on the building at the correct scale and the door handle. Use a stand-in human model in the scene and if the rest of your building follows that correct scale, you won’t have to constantly measure if the general scale is right. Another simple trick is to count how many vertical bricks make up a section of wall in real life, this way your texture matches. It’s important to get the scale correct early on since the whole city will be built using it. The road mesh, curb heights, and sidewalks, along with the first couple of buildings are really important to establish proper scale. I set my Maya scene to match Unreal’s units in centimeters so the grid units and measurements are in sync between the editor and my DCC app. I made block meshes for each of the buildings, and when I realized that I would need to make upwards of 40 highly detailed & unique buildings for one street I knew it was time to call on some external partners.
Trees
Every shot of the environment has some foliage in it. It’s an underrated, sometimes overlooked aspect of making a digital city look & feel real. It provides an important scale reference between tall buildings and the smaller details in the scene. Trees are very relatable objects in a space that break up the hard, straight lines with organic shapes and interesting shadows. I used SpeedTree and Megascans to create most of the greenery in the scene. After the trailer was finished I created a new scene and replaced all the foliage assets with ones I made using a free application called Treeit( You’ll find out why in lessons learned at the end.) There’s also a wind gizmo in the scene to provide a natural breeze so the plants can gently sway. Movement is a great way to liven up a scene and create more visual interest and realism. (Pro-tip: if you find your trees or grass in the scene disappearing at a distance due to mipmaps, adjust the clip bias of your leaf/branch textures, and voila! It’s fixed. Nice bushy plants from a distance)
Outsourcing
Since I was only working on this project in my spare time, I realized I needed some help if I ever planned on finishing. I quickly interviewed and hired a small team of freelancers (Special thanks to Jelena Ilievska, Mykola, Neekolaz Biliy, Travell, & Akshay Kumar) via Upwork to work on some props and buildings while I worked on other assets. I set up a Trello project page, sent out two example buildings I made and was able to delegate some work to the talented team. Overall, I sent 20 out of 40 main buildings to the team and around 10 out of over 100 props. I am super grateful for all the hard work from my external partners, I couldn’t have finished the project as fast without all their help. Outsourcing really worked best for me when other artists were working on standalone props or buildings and they didn’t need numerous revisions. Over the course of the project, costs, quality of delivered assets, and time became somewhat of a concern so I decided to complete the rest of the project without using more outsourced help. I simply didn’t think it was wise to spend more money on outsourcing when it went against the main goal of me taking on the challenge of figuring out how to create all the assets within the project.
Bringing the city to life!
After placing thousands of assets by hand, it was time to add the extra flourishes that make a digital city feel like a real place. There were several important elements added to the city to try and achieve this. Birds, boats, airplanes, pedestrians, clouds, wind, plants, decals, construction zones, parked cars, flags, sounds, traffic, and drivers in all moving vehicles. After closing out contracts with my external partners I focused on trying to use resources I could find on the UE4 marketplace or making it myself to keep costs down and see if I could learn a few new things.
A bustling city needs lots & lots of people
If you’ve ever seen the movie 28 Days Later or Vanilla Sky you know how weird it is to see a major city without people in it. I was a bit worried, when I started the project, about how difficult it would be for me to add hundreds of people milling about the city in a natural way but then I found a gem of a program that solves this. Anima made it so easy to populate my city with realistic digital people and customize them, I was truly surprised by how intuitive it all was. I was smiling every second while using it and my eyes lit up watching my once static city come to life. I imported a low-resolution version of my city as an Fbx backdrop into anima and began drawing paths and adding pedestrians into the scene. I then imported the Anima scene into the UE4 using the plug-in and presto!
Cars, Traffic, & Decals
The two things that almost broke my spirit during this project were:
- Creating a real-time, dynamic traffic system with thousands of ai cars with physics that obeyed traffic laws and had drivers in the seats.
- Placing thousands of road decals by hand.
Luckily, I was able to find a great lightweight starting point for the car models on the UE4 Marketplace but to make the cars work exactly the way I needed them to, I had to make some heavy modifications to the packs I purchased. First I had to make all the windows transparent using a masked material, edit the car mesh and cut out holes for a glass mesh, add a fake vehicle interior and driver, add a glass mesh, and driver model to each vehicle, separate the wheels, and rig the mesh in Maya, then export it into Unreal and add vehicle physics to over 15 cars. I then created an Ai traffic network with traffic lights and set the cars loose in the city. Even though it was initially frustrating to get the cars to behave, it was hilarious to leave my computer and come back to see this:
After adjusting to the slight learning curve of the traffic blueprint I realized it was mostly the sheer scale at which I was attempting to run these traffic scenarios that was problematic. I adjusted the collision avoidance behavior and vehicle physics then was able to have thousands of cars driving in the scene flawlessly for longer than 5 minutes, which was way more time than I needed per shot. I’m really impressed by the quality of the vehicle pack I modified and the traffic blueprint. I’d like to extend a big thank you to their creators. I’m really impressed with the Unreal community and the quality products they supply.
Decals
With the decals, I created a spline decal placer blueprint which came in super handy for placing all the road lines. I also was able to use a customizable text blueprint for the license plates of the vehicles for all the street signs.
Lights, Camera, Action!
To create the GTQ trailer I first focused on creating an accurate physical camera that had some barrel distortion on the fringes of the lens and displayed low light noise when there wasn’t enough light reaching the sensor. I followed the Quixel rebirth tutorial for a great starting point and made some adjustments to fit my scene.
After creating a storyboard for the trailer, I set up about 50 cameras around the city to capture interesting vignettes of the digital citizens going about their lives in their own digital San Francisco. I used Sequencer in Unreal to animate the cameras. I tried to use mostly prime lenses when filming and captured five to ten-second clips from my shot list. I created a LUT based on a real-world film camera and used that post-process to subtly tie the colors together. I didn’t want to go overboard on the color and lose that sunny, warm, California afternoon feeling. I also used a volumetric cloud plug-in from the UE4 Marketplace but kept it subtle.
I brought the rendered frames into After Effects and began editing the trailer. When I had a rough cut I liked, I selected music from Artlist. As I played multiple songs over the video I began writing the script for the voice-over. When I had a first draft of the script I sent it to some voice actors on Voices.com for an audition. I instantly fell in love with the voice of Jeff Rechner, he was perfect for the role. I then mixed all the sound effects for the trailer and prepared the final cut with the voice-over.
Lastly, some things you should know first
Wow, you read this far?! I appreciate your perseverance. As a reward, I’d like to share some of the most important lessons I learned during this project. Some of these I wish I knew before starting:
- The oldest aquarium fish in the world lives in San Francisco. She likes belly rubs. Okay, while that is true I was just checking to see if you’re ready for some juicy lessons learned.
- If you use Megascans, SpeedTree, or any assets you don’t fully own distribution rights to, all those assets will have to be removed from the project if it is to be used in the UE Marketplace. Rendering stills and videos of your work with those assets are fine but make sure you create your own assets if you’d like to avoid any licensing conflicts. Consult Epic for more information.
- If you are working on projects in your free time while employed make sure you talk to a few good employment lawyers who are well versed in the employment laws of your State. From my experience, some companies don’t care if you work on stuff in your free time, some companies actually encourage moonlighting, and others will claim they own every idea, project, asset, you ever create even in your free time, as long as you are employed by them. Make sure you know which type of employer you’re working with and talk with legal experts if you have any questions. I am not a lawyer and none of this is legal advice. I’m simply an artist who likes making art in my free time.
- Buy a power backup in case of power surges and outages. Losing work is no fun!
- Back up your Unreal project often. Near the end of this project, I accidentally moved and saved over 100 building positions that were in a shared level. I couldn’t undo it, because I saved then quit, but luckily I had saved a manual backup of the level and only lost about 10 mins of work.
- Sweat all the details. The micro details make up the bigger picture.
- Make sure to remove branding in textures and props from the beginning. it helps to make up a bunch of cleared fictional brand names early so you're not spending a whole evening removing them from hundreds of textures and meshes.
- Do a serious optimization pass. Try to do this as you import textures and meshes into Unreal. It’s no fun doing this all at the end.
- Pay attention to lighting, light is the single most important element of displaying realism, if you need proof just close your eyes.
- Know the value of your work and your time.
- Artists generally don’t make whole city scenes by themselves (unless they’re looking for a tough challenge) because it’s extremely hard and it takes a long time. It takes an even longer time to do it well. it’s usually a team effort.
- If your goal is to build large-scale realistic cities doing it all by hand is not the most efficient way forward(I knew this before starting). I’d recommend working with the best Houdini procedural artists out there. Specifically technical artists who have a solid artistic eye and really care about matching realism as closely as possible. They will also be able to quickly build tools to help unlock an art team’s full potential allowing them to pump all the detail directly into the scene.
- Now I know what’s taking GTA VI so long to be finished, I can’t wait!
- Always add birds and airplanes in the sky for scale and motion. Airplanes also help suggest it’s a lively city with people coming and going. It’s subtle but only a few games do this including Super Mario Sunshine.
- A global pandemic can be the perfect opportunity to get a lot of work done on your passion project and not feel guilty about turning down that invite to a concert, club, or dinner party your friends keep inviting you to. The last two years have been tough for people around the globe, having a long project to work on can keep your mind on more positive, creative things.
- Take breaks! The last thing you want to do is burn yourself out.
- Keep an Instagram page for your art. You can find more in-progress screens and videos there.
- Lastly and perhaps one of the most important lessons is to have fun with it! Give your brain a chance to fall in love with learning and the process of failing forward till you get really good at what you’re trying to do. It’s an integral part of the process and will make you a better artist. Don’t give up, you can do it, you can do anything you put your mind to especially if you are willing to persevere!
Great news! AK has decided to make his copy of this massive project available on the Unreal Engine Marketplace! Visit Here for more details!
Resources:
- Unreal Engine 4.27.2
- Anima
- TreeIt
- Speedtree
- Megascans
- Substance Painter
- Maya
- After Effects
- Voices.com
- Upwork
- Trello
- Arch vis traffic system
- Cloud Plugin
- Birds plugin
- Artlist.io
- Parked cars blueprint and 7 detailed cars pack
- Vista building city generator
- Backblaze
- Factor75 meals
- A very special thanks (in no particular order) to Akura, Mom, Dad, Ernesto, Gil, Paul, Karlo, Mitzi, Ilona, Sean, Joe, Rachel, Matt, Ed, Larry, Doris, Dino, Meghan, Simon, 80.lv, 3dnchu, & anyone else I may have left off this list.
