Win3mu Part 10 — Leveraging Reflection

This is Part 10 in a series of articles about building Win3mu — a 16-bit Windows 3 emulator. If you’re not sure what this is or why I’m doing it you might like to read from the start.

In a previous post about the Module Loader I gave an example of what’s involved in calling a Windows API method from VM code. In that example I showed how parameters are read from the VM stack and type converted, the real Windows API called and then how the return value is passed back to the VM.

All this parameter and type conversion code is not only tedious to write but error prone if done by hand. This post explains how Win3mu leverages C# reflection to automatically do the heavy lifting.

What is Reflection?

Reflection is a mechanism by which a program can inspect itself. For example in C# all of a type’s methods can be found like this:

foreach (var mi in typeof(SomeClass).GetMethods())
{
Console.WriteLine(mi.Name);
}

Similarly, a method’s parameters can be discovered:

foreach (var pi in mi.GetParameters())
{
Console.WriteLine(pi.Name);
}

Also important to this discussion are Custom Attributes. Custom attributes are used to decorate code with additional information that can be retrieved at run time (via reflection).

For example, here is a method decorated with a custom attribute called “EntryPoint”:

[EntryPoint(0x0001)]
public void MessageBox()
{
}

In Win3mu all emulated modules (eg: Kernel, User, GDI etc…) derive from Module32 whose primary responsibility is to use reflection to provide the plumbing between the VM and the API methods that each module provides.

Creating API Thunks

The first thing Module32 does is create call thunks for each of the methods that the module exports. Exported API methods are decorated with an “EntryPoint” attribute that declares the 16-bit ordinal of the exported method.

Any method with an EntryPoint attribute is now callable from 16-bit code.

Handling Parameters

The next task is to lift the parameter values from the VM stack and pass them to the C# function. Module32 looks at the function’s parameters (using reflection) and works out how to get these parameters from the VM stack.

The following code shows how two simple types “ushort” and “string” are read from the calling VM stack and passed to a C# function:

Although not shown in the above example, the return value is handled similarly by inspecting the method’s return type and updating the ax (or dx:ax) register after the function has been invoked.

The MessageBox function can now be implemented like this:

This is starting to look a lot cleaner. (Compare this to the original attempt).

Type Widening and Narrowing

If you look closely at the above example you’ll notice there’s no actual code — it’s essentially one function directly calling another and its only purpose is to convert the options parameter from ushort to uint and the return value from uint to ushort.

This widening and narrowing of types is prevalent throughout the set of API methods that Win3mu needs to support so it’s worth including some automatic support for this through two new types “nint” and “nuint”.

I stole these names from Xamarin for iOS where the “n” stands for “native” (as in native to the platform on which it’s running). In Win3mu a nuint is a 16-bit on the 16-bit side and a 32-bit on the C# side.

Mapped Types

Originally Module32 included direct support for nint and nuint but it didn’t take long to realize that there were quite a few types that needed similar special handling. Rather than hard code support for each of these into Module32, it now looks for types declared as mapped types.

Take a closer look the nuint type above. Notice the [MappedType] custom attribute? This tells Module32 that it’s a type that needs some special handling when transitioning between 16 and 32-bit worlds.

Mapped types must have two static methods named “To32” and “To16” which will be called to do the conversion. You can see these two methods in the above example.

Win3mu has MappedTypes for many of the Windows API types. eg: POINT, SIZE, RECT, HWND, HDC etc…

Revisiting MessageBox

Applying all of the above reduces the MessageBox function to just its declaration. No actual code!

Other Parameter Types

There are a few other cases that Module32 can cope with like ref and out parameters, special handling for output strings, structure types and a few other edge cases.

Of course not every function can be directly mapped onto a Windows implementation but there are large sections of the Windows API where this works really well.

GDI for example has many functions that can be directly mapped while Kernel has only a few — since most are implemented by Win3mu.

In cases where the automatic mapping doesn’t work it’s easy to fall back to a simpler type and handle it in the method implementation.

For example, the TextOut method has a string parameter whose length is passed in another parameter. I could add support to the type mapping for this, but it’s a rare case so I declared the string parameter as a uint (the same size as a 16-bit far string pointer) and wrote code to handle it specially:

What About Performance?

Reflection makes it really easy to setup these bindings between 16 and 32-bit code. The downside is that performance isn’t great:

  • MethodInfo.Invoke() is well known to be slower than calling a function directly from C# code.
  • Every time an API method is called the parameters need to be re-investigated, stack and type conversion look ups done, an array created for the parameters and then similar handling for the return value. There’s a lot of overhead on each and every call.

So far I’ve only noticed performance issues with one program. There’s a Checkers program (by Gregory Thatcher) makes about 50,000 calls to LocalAlloc and LocalFree when it’s planning it’s next move. That’s 100,000 API calls which takes less than a second under DosBox but about 6 or 7 seconds on Win3mu.

(I double checked — it’s not a problem with the local heap implementation. Making a similar number of calls from C# code took just a few milliseconds so this is probably related to reflection and/or perhaps just slower CPU execution — but that’s another area that hasn’t been performance tuned yet).

I’m not too worried about this because it can be fixed later by generating dynamic methods. Dynamic methods are a way to generate code (MSIL) at run-time that’s compiled to machine code when executed.

Rather than inspecting a method’s parameters each time the method is executed, eventually Win3mu will look at the parameters once and generate a dynamic method that knows how to do all the conversion. Not only will that code be compiled to machine code, it won’t have to re-investigate the parameter types each time since that information will be baked into the generated code.

I used this approach in PetaPoco (a tiny database ORM) with excellent results — it gives performance close to that of compiled code with all the advantages of reflection.

Writing dynamic methods is a little like writing assembly language (see here for an example). This is why Module32 doesn’t try to handle every different kind of parameter mapping. By keeping it simple, things will be easier when it comes time to dynamically generate code.

Progress

Recent progress on Win3mu has been held up by Visual Basic 1.0. Getting VBRUN100.DLL to work under Win3mu is proving to be extremely difficult. I’ve set it aside a couple of times but it’s bugging me that it doesn’t work.

The main problem was that it assumes knowledge about the structure of the local heap that was incompatible with the way Win3mu’s heap works. It was quite a task to figure this out…

I now have the heap issue resolved but there are other problems and it still doesn’t run. I’ve set it aside again while I beef up Win3mu’s debugging capabilities.

End of Series

I’ve decided to end this series of articles here. So far the topics covered have been building on each other and really needed to be read in order. It made sense to group them into one series.

Now that the core of how it works has been covered from here on I’ll be writing shorter series (1 to 3 parts each) covering topics such as:

  • implementing the Windows API (including the messy business of Windows messages) (read here)
  • the debugger (read here),
  • shell integration and user experience,
  • details on problems with particular programs,
  • performance tuning,
  • other areas of interest that come up — let me know what you’d like to read about.

To those who’ve been asking about trying it… unfortunately it’s still too soon. Most program’s still don’t work and tend to explode spectacularly. To make it available now would mostly lead to disappointment.

Also, I’m not entirely sure what I’m going to do with this project. It started out as a personal challenge/learning experience but I’ve since spent a lot of time on it (probably too much time). As an indie dev it’d be nice to get a little return on it to justify all that time but I haven’t decided anything yet.

I hope you’ve enjoyed this series and that you learned something interesting. I know I have!


Hi, I’m Brad Robinson — an independent software developer living in Sydney Australia. I write software for musicians and as an indie developer I rely on word of mouth.

If you enjoyed this article please consider sharing it by hitting the “recommend heart” below or by sharing on Facebook/Twitter. It’s a small gesture but makes a real difference.

Also, if your feed is lacking in hex dumps, disassembly listings and screen shots of old Windows 3 games you might like to follow me on Twitter.

Continue reading… The Debugger or Implementing Window Messaging.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.