How to make your own game engine (and why)
So you’re thinking about making your own game engine. Great! There’s lots of reasons to want to make one yourself instead of using a commercial one like Unity or Unreal. In this post I will go over why you might want to, what systems are needed in a game engine, and how you should approach development of it. I won’t be going into any deep technical details here, this is about why and how to develop a game engine, not a tutorial for how to write the code.
Lets start with the absolute first question you should be asking yourself if you want to make your own game engine: Why?
Here’s a few good reasons why you might want to:
- Novel Tech: You want to make a game that uses a piece of novel tech that no other engines out there currently support, or can’t be easily made to support in their current state. This can mean some kind of massive-scale simulation that requires some to-the-metal coding to make performant (Factorio) or some custom thing that doesn’t fit into any existing molds (Noita, Miegakure), or wishing to target an odd piece of hardware that current engines don’t support (Playdate), or any number of other things really. It’s a good reason to make your own engine, cause there isn’t any other option in cases like this.
- Specialization: You want to optimize your workflow for the kind of games you make. You don’t need all the features included in a commercial game engine, and you can make your asset pipeline / level editor / whatever way smoother to use when considering your specific use cases instead of needing it to be general purpose. Specialization is almost a requirement of making your own engine, if you aren’t specializing it and catering it towards your exact use case, you should rethink why you’re making an engine in the first place.
- Independence: You don’t want to be dependent on someone else’s technology in the long term. The incentives and values of a company like Unity or Epic are not always going to align with your own, and you want control over your own tech, the ability to fix bugs yourself instead of “waiting and hoping”, and comfort knowing that an update won’t completely break your current project. You are willing to eat the cost of developing your own tech, because in the long run its good to not have to constantly change stuff depending on the whims of giant companies.
- Curiosity/Learning: You’re just curious about how it works and why other engines have made certain decisions they did. This is a great reason, one of the best reasons actually, to make your own game engine.
Also while I’m making lists, here’s a couple of bad reasons for why you would want to make your own game engine. If any of these are your (only) motivation, you should back up and reconsider:
- I can do it better: You think you can just make something better than Unity or Unreal (or Godot or GameMaker) in general. You can’t. It’s possible to make something that is better than these for specific use cases (see: Specialization above), but you, as an individual or a tiny team, are not going to compete with these for general purpose stuff. Especially if you have never made your own game engine before.
- It’s the what real programmers do: There is no “right way” to make a game, you don’t win programmer points for making your own engine. If your game is a good fit for an existing engine (and I’d argue like, 99% of games are), there’s no shame in using one. After all, a game engine is simply a tool for which to make a game, its not and should never be the goal by itself. The games you make with it are all that matters.
- Saving Money/Time: You most likely won’t. Making an engine takes time, and time=money. Unity’s cost is negligible compared to the time it takes to make your own tech. Unreal’s rev share is negligible unless you’re selling 10 million copies of a AAA game. Using your own engine won’t make you sell more copies of your game automatically. And while you *can* save time in the long run, this usually means having your engine be good enough to carry you across multiple projects, while also providing you with significant workflow improvements compared to commercial engines. It’s not easy to get this right, and you definitely won’t if its your first try at it (and extremely unlikely if you’re doing 3D instead of 2D).
Also, you should absolutely consider your own experience and goals when weighing all of this. If you don’t have a lot of experience making games, you will have a much harder time making an engine (and you should definitely go get some game making experience before trying to make an engine anyway). If you want to do 3D, you will have a much harder time than just doing 2D.
For me, the motivations to making my own engine are mostly about novel tech and specialization. I come from a flash game background, having made quite a lot of them in the mid/late 00s, and its a workflow I am really comfortable with and enjoy. None of the existing engines out there supported importing flash animations, so my only option was to do it myself. It’s extremely nice to just be able to plop .swf files into my resources folder and instantly have the animations available for use in my game code, without needing any middle steps to export them to sprite sheets or whatever.
These aren’t the only reasons why you’d want to make your own game engine. There’s a ton of advantages (and disadvantages) to it, and whether or not the pros outweigh the cons depends a whole lot on how much experience you have with both game dev in general and lower level coding. Like, it’s nice to just have your own tech, and its nice to not have to constantly google for tutorials that might be outdated for how to do something in your engine. It’s nice to be able to actually debug the internals of your game if something goes wrong. But it can also suck if you made a couple of bad design choices and everything falls apart entirely, and there’s no resources online to help you. You have full control and full responsibility, and all the pros and cons that come with that.
Before we get into how to make your own game engine, lets take a step back and define what a game engine actually is, and the types of problems a game engine is supposed to solve.
A game engine is the framework upon which you make a game. It provides you a foundation on which to build on top of, and all the building blocks and lego pieces you need to assemble a game out of. It provides a boundary between “game logic” and “boring technical stuff” so that your game logic code (the stuff that actually describes what your game is, how it controls, the interactions between objects, and the rules), doesn’t need to actually care about stuff like how to send a triangle to the graphics card.
There’s actually a lot of variance in how much game engines actually do for you. Some of them are barely more than just a framework for displaying graphics, doing very little for you beyond that (Flash, Pico-8). Some of them are basically an entire game by themselves, or at least hyper-specialized for a specific genre, putting a ton of common game logic in the engine itself (RPGMaker, Ren’Py). And everything in-between.
Game engines themselves also tend to be built on top of even lower level frameworks like SDL and OpenGL, and include many special purpose libraries for things like audio, physics, math, and whatever else is out there that you find useful. Making a custom game engine does not mean writing every little piece of it yourself, especially something as standard and useful as SDL. Assembling the right set of existing libraries for your use case is part of making an engine as well, and there do exist libraries out there for nearly all of the systems you could want in your engine.
Anyway with that in mind, lets talk a bit about what I consider the most basic feature set you need for your game engine. The bare essentials. The minimum that you need before you can start making a game.
- System Initialization: opening a window, obtaining an OpenGL/DirectX/Vulkan context, initializing audio. SDL will handle all of this for you, so just use SDL. Like really, SDL is industry standard at this point for custom engines, there’s no reason to do this part yourself.
- Frame Timing Control: You want your game to run 60fps, so you need some kind of timer and loop that controls when updates and renders happen.
- Input: You need to be able to respond to button presses. There’s a lot of different ways to respond to button presses, maybe you just want to be able to query the current state of a button, or maybe you want to register events, it doesn’t really matter. SDL will report button inputs to you, if you’re already using that for system initialization you should be using it here as well. You can build a really powerful and flexible input system on top of this, but its not necessary to do anything more than the basics at first.
- Rendering: Most, probably at least like 75% of games, use graphics in some way, and this is absolutely something that should be your engine’s responsibility. If you’re doing a 2D game, the bare minimum renderer just needs to be able to display simple textured quads on the screen. Shaders, vertex buffers, render targets, meshes, materials, and whatnot are all great, but they can come later as you need them. If you want to get your hands dirty with OpenGL or Vulkan or whatever you can absolutely go ham with your custom renderer, but again there’s no shame in using an existing library like Ogre3D to cover rendering. Entirely depends on your goals and your needs and what problems you actually find interesting to solve.
- Math & Misc. Utilities: I don’t really consider “vectors and matrix math” a full on “System” but it is something you should probably have as a utility library that engine and game code both have access to. Plus any other random useful functions and formulas you find as you dev. STB is an incredibly great resource for random utility stuff you might need.
Also here’s a few systems that are bare essentials as well, but you don’t need to add them until later on when you actually need to start using them.
- Game Object / Scene Management: Coding everything in one big update function isn’t actually as bad as it seems, but if your game starts getting even a little bit complex you’re going to want some kind of system to handle individual game objects and collections of game objects. In a way this ends up being the most important system in your engine, as it is what drives what your game logic looks like. Are your objects big inheritance trees? Are they made up of smaller components? Are you using ECS? (sidenote: you probably shouldn’t use “pure” ECS unless you have a really specific use case for it) What types of events do they respond to? How do you query for other objects to interact with? Do you care about memory layout? These are all things that your game object / scene management system should cover. There are some existing libraries for this, especially for pure ECS, however since this is the system that has the most impact on what your game code looks like, I heavily lean towards “do it yourself” here. Relying on an existing solution will force you to constantly think about how to fit your game logic into that mold. Exactly backwards from what you’d usually want. You want your engine to be designed to support the way you want to express game logic, not the other way around.
- Audio: Sound Effects and Music are kind of separate systems here, though both lumped under “Audio”. Basic functionally you need is the ability to start and stop audio loops (music) and the ability to play one off sound effects in their entirety. There’s a whole lot more to audio than just this, but you can get very far with just the basics here. The annoying thing about audio is that the industry standard audio frameworks (FMod and Wwise) are both commercial and come with a bunch of licensing restrictions, but a lot of the open source ones are just really annoying to use (shoutout to OpenAL). Personally I find FAudio pretty simple and pretty great, and I use that as a base to build more complicated behavior off of.
- File Loading & Management: All games need to load files. You need a way to load files. You probably don’t want to redundantly reload/decode files that have already been loaded in, so you need some kind of basic manager that handles all of this. You might want to be able to build mod support or dynamic loading/unloading or whatever off of this, as long as you have the basics here and only ever load files through the file manager, you can easily add whatever other functionality you want into this later. You don’t need this immediately, since you can just use built in file loading as a placeholder, but you will want to eventually have a file / resource manager system in place once you start using files more often.
- Networking: Ok, networking (online multiplayer) is VERY optional (if you don’t need online multiplayer, do not bother), however its an exceptionally difficult system to staple onto an engine that was not designed with multiplayer in mind, so if you are making a multiplayer game you need to consider this early because its mere existence influences the design of literally every single other system.
That is the basic set of systems that you’d need to have what I consider a game engine. Other systems like Collision and Physics and Serialization and Animation and UI and whatnot are actually optional. Most engines have them because they’re common enough that its worth including, but you don’t need those to make a game. You can get away with simple collision checks in your math utility library, and just let game logic do that all manually. You can do basic gravity and acceleration without a physics engine like Box2D or Bullet. And full serialization can often be huge overkill if all you need to do is save which checkpoint you spawn at.
If you’re writing your own engine, you will necessarily have less systems and features in it than the big general purpose commercial ones have. That should be the goal! Unity and Unreal are bloated monoliths, any individual game is only using a small slice of what they have to offer. Add only what you need, for your specific use case, and focus on making what’s important to you and your game extra good.
I very strongly recommend becoming familiar with how a bunch of different game engines work before starting out making your own. Learn what kinds of paradigms and structures they use, what’s cool about them and what’s annoying about them. Make a small game in a bunch of different engines if you can, just to obtain that knowledge.
Ok so you’ve weighed the whys and explored the whats, and decided that you want to make your own game engine. How do you go about doing this?
I’ll get straight to the point: Make a game at the same time as you’re making the engine. This is an unbreakable rule. The only unbreakable rule. Get the basics in as fast as you possibly can and then immediately start making a game on top of it. An engine is nothing without a game.
You need to do this, because the features of an engine should be informed by the needs of the games made in it. You will not understand how to architect a good animation system if you don’t have a game that needs complicated animation flow. You will not know what your performance bottlenecks are if you don’t have a game. Do you need some kind of tree based world structure that will avoid updating or rendering objects that are too far away from the camera? I don’t know, and you don’t know either, until you have a large level that runs like crap. Even then, updating the objects might not be the actual bottleneck and you wont know until you profile it.
Don’t code something until you need it. If the only UI in your game is a play button on the main menu, congrats! You don’t need to code a fancy UI system for your engine. The End Is Nigh shipped without a physics engine, or even a collision broadphase. Hell it didn’t even have a camera system, since the game just didn’t need it. I used a .csv spreadsheet to arrange the world map instead of some complicated editor. It was easy and worked fine.
I’m not going deep into implementation details for these systems because there’s just way too many ways to do it that all work just fine for different purposes. There is no “best way to do rendering” or “best way to do game object management”. It all depends on your game. Start with the basics and expand them as you find need to.
For programming languages, use what you’re comfortable with. Making an engine is a big involved task, if you also try to learn C++ at the same time as trying to learn how to make a game engine you’re just doubling the difficulty of learning either of those tasks. C# is perfectly fine for making a game engine. Slower than C++, but often not slow enough to matter. Something really slow like python might be a bit of a stretch if your game has a lot of moving parts… but for some kinds of games it’s still fine. Use what you’re comfortable with.
Also, you will never get this right on your first try. My first game with a custom engine was Closure, and it was an absolute mess (despite, kind of hilariously, being nominated for a Technical Excellence award at the 2010 IGF). One big update function and one big render function handled basically the entirety of the game. Adding new kinds of game objects was extremely annoying and required adding code in a ton of different places, and working with some janky ass custom editors for animations, so we ended up with only about a dozen different kinds of interactive objects in the end, though some of them had some variants (bool toggles) that changed their behavior significantly because it was easier to add variants than it was to add new objects. The spotlights, mirrors, and turret are all the same object!
But you learn from this experience. Closure’s engine was a mess, but it still shipped in that state, and it was “good enough” that I could run it on the PS3 without that much hassle. It was tempting to rewrite the engine at points, but rather than do that (which would have only served to delay the game), I just kept notes of all the things about it that sucked so I could do things better the next time. Especially things that got in the way of actually making the game. I did the same thing with The End is Nigh. Its engine (while much, MUCH better than Closure’s) still had a bunch of issues with it that I just grit my teeth and dealt with during dev. As soon as the game came out, I immediately started updating the engine for use in our next project, fixing all the annoyances with it and adding in the new features we needed.
It’s an iterative process through and through. You learn, make a game, iterate and repeat. Over and over until eventually your engine is good.
Hopefully you should understand why I didn’t bother giving any technical details for how to implement all those individual systems. It just depends way too much on your specific use cases, and there’s hundreds of different, completely valid ways, to approach each of them. Figuring out what works for you IS what making your own game engine is about, and that’s the mindset you should be in when writing your own stuff.
Anyway that’s about all I wanted to cover in this post, hopefully you’re either motivated to write your own game engine now, or scared away from the thought of it entirely. Both are good outcomes! If you have any actual questions about technical details or opinions on game dev or game engine development, feel free to ask me about it on Twitter (publicly) and I’ll do my best to respond. Thanks for reading!