Mana Engine: Thread safety of APIs

In the previous post, I talked a bit about how we maintain thread safety among our own data. The game components.

That’s great, but sometimes you aren’t entirely in control of the data you need to work with. The common example of this are APIs and Libraries. Either in-house or third-party, APIs and Libraries (I’ll just call them “API” from now on) are often entirely self contained or have their own memory allocation and access patterns that may or may not be thread-safe.

Sure, you could just put a lock or mutex around any access to the entire API, but that’s how you get yourself into lock-hell and really terrible performance.

In the Mana Engine, we wanted to make use of some external APIs. Notably, Ocornut’s Imgui was an early one we adopted because of it’s usefulness. We quickly found, however, that Imgui is not thread safe. And really, I didn’t expect it to be.

But we have this task-based multi-threaded engine. How do we make sure we’re accessing the API in a safe way?

Using Imgui is split up into 3 primary parts. First, you start the frame. Second, you add controls. Finally, you render the controls.

//Step 1:
ImGui::NewFrame();
//Step 2:
ImGui::Begin("My Window");
// Add some controls in your window.
ImGui::End();
//Step 3
ImGui::Render();
//Then call ImGui::GetDrawData() and push the vertices to the GPU.

To make this work, we need to have one system handle the NewFrame call (along with a bunch of inputs and other setup for the frame).

Another set of systems will create the controls they need. Right now I have two systems creating controls because I have two windows. Because of thread-safe limitations, I can only access the API from one of these two systems at a time, so they need to be run in serial.

Then finally one system calls ImGui::Render() and handles the GPU stuff.

I haven’t talked about Singleton Components yet, but they’re a feature of the Mana Engine where you know you’ll only have one of a thing (as opposed to normal components, where you expect many and they are each bound to an entity). Singleton Components are still keyed into Authorized Systems and have similar rules as regular components with respect to read/write access.

Of course, there’s nothing saying that the Singleton Components need to actually have any data. They can totally be just an empty struct that simply serves to create dependencies between systems. To guarantee thread safe access to the ImGui API, we do just that. A graphics programmer could think of these simply as fences.

We came up with a macro that creates a “token” singleton. It’s basically just a regular Singleton Component with nothing on it. By asking for const access to that Singleton Component, a system is guaranteed to run after any writers. Because we have three “layers” of dependencies, we’ll need two tokens.

DEFINE_TOKEN_COMPONENT(ImGuiBeginToken);
DEFINE_TOKEN_COMPONENT(ImGuiControlsToken);

Then the system that starts the frame looks something like this:

class ImGuiBeginFrame : public AuthorizedSystem<ImGuiBeginToken>
{
    void Update()
    {
        ImGui::NewFrame();
        //Also pump in some inputs, window size, etc.
    }
};

While any system that adds controls looks like this:

class DebugWindow1 : public AuthorizedSystem< const ImguiBeginToken, ImGuiControlsToken>
{
    void Update()
    {
        ImGui::Begin("Window1");
        //Add our controls.
        ImGui::End();
    }
};

Finally, the rendering system:

class RenderImGui : public AuthorizedSystem<const ImGuiControlsToken>
{
    void Update()
    {
        ImGui::Render();
        ImDrawData* pData = ImGui::GetDrawData();
        if(pData)
        {
            // Send data to GPU.
        }
    }
};

Because the DebugWindow1 system wants const access to the ImGuiBeginToken, it’s guaranteed to run after the ImGuiBeginFrame system. Also, because it has write access to ImGuiControlsToken, it will always run in serial with other system that have the same access. To end it all, the RenderImGui system has const access to the ImGuiControlsToken, guaranteeing it always happens after all controls have been created for that frame.

The frame graph ends up looking like this. A single line. Which is perfect for what we want. Tasks can be one whatever thread they happen to get picked up on and no two threads ever touch the API at the same time.

There is a small bit of nuance here. Technically, DebugWindow2 could go before DebugWindow1 in our engine. It all depends on the order they get initialized in. Without going into that, suffice it to say that there are plenty of mechanisms for controlling the order DebugWindows run in.

So this works out great as a lock-free way of integrating an API that is not thread-safe into our task-based engine. While it will never wait or block, it is processing a lot in serial that could potentially be parallel if ImGui supported it natively. However, this is pretty good and the approach has worked with other APIs we’ve used. Moreover, updating the UI is not our performance bottleneck, so making ImGui threadsafe is a goal driven by safety — not performance. Being run in serial isn’t really a problem at all.

You will notice about this solution is that what I’m trying to tell the engine is “I want SystemA to depend on SystemB” without explicitly calling out SystemB. It’s almost like I want a syntax that goes something like DependsOn<SystemA>. The issue there is that in the Mana Engine, a key philosophy is that no system needs to have knowledge of another system. Systems only have knowledge about components, and that decoupling it what makes the engine so modular.

On the other hand, the token setup is nice, because I can continually add more and more systems to be writers to ImGuiControlsToken and the RenderImGui system never needs to know about those changes. There’s real flexibility and power in that, especially if you want to insert a new system into an existing set of systems that maybe you didn’t write. (In fact, I’ve already done this sort of thing by allowing my terrain system to “hook” into the rendering systems in this very same way).

Lastly there is this parallel to graphics programming — a place where highly parallel and concurrent work gets done. These tokens can just be thought of as fences or semaphores, where systems can indicate which side of the fence they need to be on. Being able to tap into existing patterns game developers will be accustomed to is important for adoption and ease of use.

That wraps it up for this little peak into the Mana Engine. I know these posts are not comprehensive and a bit out of order in regards to introducing new mechanics about the engine. If I were writing something like coursework or a tutorial series, I would have introduced Singleton Components earlier. So bear with me if it’s a bit hard to follow — I apologize. But if you want to know more or have questions, leave a comment or @ me on twitter (tloch14) and I’ll be happy to answer any questions you have.