OscMapper for Unity

Automatically mapping an OSC controller to a ScriptableObject in Unity

Every now and then I have the need for controlling my application from an OSC controller (in my example I’ve used an iPad as a controller). I wanted to be able to write a ScriptableObject and then match its fields and properties with OSC addresses automatically, so that whatever data came in over network would update said fields and properties. That way, I could just drag and drop a OSC mapping component into my scene, set a reference to a ScriptableObject, then use the inspector to set up the actual mappings rather than tediously wire up addresses, fields and properties.

This article describes the code to create such a component, with an accompanying editor. I’ve used the term mapping, but if you prefer binding you can do a mental substitute if that makes you happy.

Since I specifically wanted mapping to OSC that’s what this article is about, but hopefully you find this code useful for other types of ScriptableObject mapping

Low-Level OSC

Rather than roll my own OSC library, I opted for UnityOSC. It does both sending and receiving, and is licensed under the MIT license.

For actually sending OSC messages to my application I went with TouchOSC as an OSC controller application. It’s fairly cheap and has an editor for Windows/MacOS which syncs layouts seamlessly with Android and iOS devices.

With the low-level OSC stuff out of the way, let’s dive into the mapping process.

Creating the mapping component

First we need to be able to create each mapping. You could of course extend OscType to be able to map more types, or eventually swap it for Unity’s SerializedPropertyType

public enum OscType
{
    Float,
    Int,
    String,
    Vector2,
    Vector3,
}

[Serializable]
public class OscVariableMapping
{
    public string Address = "";
    public string Variable = "";
    public OscType Type = OscType.Float;
}

Then we need to be able to do a bit of reflection to get the variables we want to map. I opted for tagging the variables with a custom attribute, so that only the variables I wanted would be exposed to the OSC mapping.

using System;

[AttributeUsage(AttributeTargets.Field | AttributeTargets.Property)]
public class OscMappable : Attribute
{
}

Next up came some convenience utilities for extracting property fields from a Type

public class PropertyUtils
    {
        public static string[] GetPropertyNames<T>(T objectToInspect, bool onlyPublic = true, IEnumerable<Type> requiredAttributes = null)
        {
            return GetPropertyNames(objectToInspect.GetType(), onlyPublic, requiredAttributes);
        }

        public static string[] GetPropertyNames(string typeName, bool onlyPublic = true, IEnumerable<Type> requiredAttributes = null)
        {
            return GetPropertyNames(Type.GetType(typeName), onlyPublic, requiredAttributes);
        }

        public static string[] GetPropertyNames(Type objectType, bool onlyPublic = true, IEnumerable<Type> requiredAttributes = null)
        {
            var bindingFlags = BindingFlags.Instance | BindingFlags.Public;
            if (!onlyPublic)
                bindingFlags |= BindingFlags.NonPublic;
            
            return objectType
                .GetProperties(bindingFlags)
                .AsEnumerable()
                .WhereIf(
                    requiredAttributes != null,
                    f =>
                    {
                        var customAttributeTypes = Attribute.GetCustomAttributes(f)
                        .Select(a => a.GetType());

                        return requiredAttributes.Intersect(customAttributeTypes).Count() == requiredAttributes.Count();
                    })
                .Select(f => f.Name)
                .ToArray();
        }
    }

I created another class for mapping fields, with the only difference being the objectType.GetProperties() was swapped for objectType.GetFields(). You could of course simplify this by creating a unified class and passing in flags for whether to get properties, fields or both.

The WhereIf LINQ extension is as follows (since I prefer method syntax over query syntax)

public static class LinqExtensions
    {
        public static IQueryable<TSource> WhereIf<TSource>(
            this IQueryable<TSource> source, bool condition,
            Expression<Func<TSource, bool>> predicate)
        {
            if (condition)
                return source.Where(predicate);
            else
                return source;
        }

        public static IEnumerable<TSource> WhereIf<TSource>(
            this IEnumerable<TSource> source, bool condition,
            Func<TSource, bool> predicate)
        {
            if (condition)
                return source.Where(predicate);
            else
                return source;
        }
    }

Because fields and properties are not the same thing, I added a small struct for indicating whether a mapping was one or the other.

public struct MappingVariable
{
    public string name;
    public bool isField;
}

Now we can get to the meat of things. The actual OscMapper component. This class uses all the code above to create the mappings needed.

[ExecuteInEditMode]
public class OscMapper : MonoBehaviour 
{
    public OscConfig OscConfig;

    public ScriptableObject ObjectToMap;
    public List<OscVariableMapping> Mappings = new List<OscVariableMapping>();
    public bool MapOnlyWithAttribute = true;

    [HideInInspector]
    public MappingVariable[] MappableVariables;

    private OSCServer listener;

    private const string TRANSMITTER_NAME = "OSCTransmitter";
    private const string LISTENER_NAME = "OSCListener";

    void Start()
    {
        if(Application.isPlaying)
        {
            OscConfig.LoadConfig();

            OSCHandler.Instance.Init();

            OSCHandler.Instance.CreateClient(TRANSMITTER_NAME, IPAddress.Parse(OscConfig.OscHostAddress), OscConfig.OutgoingPort);
            listener = OSCHandler.Instance.CreateServer(LISTENER_NAME, OscConfig.IncomingPort);
        }

        FetchMappableVariables();
    }

    void Update()
    {
        if(Application.isPlaying)
        {
            for(int i = 0; i < OSCHandler.Instance.packets.Count; i++)
            {
                var packet = OSCHandler.Instance.packets[i];

                HandlePacket(packet);
                OSCHandler.Instance.packets.Remove(packet);
                i--;
            }
        }
    }

    public void FetchMappableVariables()
    {
        if(ObjectToMap != null)
        {
            Type[] requiredAttributes = MapOnlyWithAttribute ? new[] { typeof(OscMappable) } : null;
            var fields = FieldUtils.GetFieldNames(ObjectToMap, requiredAttributes: requiredAttributes).Select(n => new MappingVariable { name = n, isField = true });
            var properties = PropertyUtils.GetPropertyNames(ObjectToMap, requiredAttributes: requiredAttributes).Select(n => new MappingVariable { name = n, isField = false });
            MappableVariables = fields.Concat(properties).ToArray();
        }
    }

    private void HandlePacket(OSCPacket packet)
    {
        if (packet == null)
            return;

        foreach(var m in Mappings)
        {
            if(m.Address == packet.Address)
            {
                switch(m.Type)
                {
                    case OscType.Float: 
                    case OscType.Int:
                    case OscType.String:
                        {
                            SetValue(packet.Data[0], m.Variable);
                        } break;
                    case OscType.Vector2:
                        {
                            SetValue(new Vector2((float)packet.Data[0], (float)packet.Data[1]), m.Variable);
                        } break;
                    case OscType.Vector3:
                        {
                            SetValue(new Vector3((float)packet.Data[0], (float)packet.Data[1], (float)packet.Data[2]), m.Variable);
                        } break;
                    default: break;

                }
            }
        }
    }

    private void SetValue(object value, string targetName)
    {
        try
        {
            MappingVariable target = MappableVariables.First(mt => mt.name == targetName);
            if(target.isField)
            {
                ObjectToMap.GetType().GetField(targetName).SetValue(ObjectToMap, value);
            }
            else
            {
                ObjectToMap.GetType().GetProperty(targetName).SetValue(ObjectToMap, value);
            }
        }
        catch (InvalidOperationException)
        {
            Debug.LogWarning($"Tried to set an OSC mapping target value for nonexistant targetName: {targetName}");
        }

    }
}

There are quite a few things going on here. Of special interest is HandlePacket() where you can write specific implementations of the OscType you might want to create. Notice that in FetchMappableVariables() I specifically set OscMappable as a required attribute. You could add any custom attributes you want here.

The Editor

To make this component easy to use i wrote a custom editor for it. I used Catlike Coding’s Custom List tutorial as a basis for the following class which handles the list functionality of the mapping editor.

public static class MappingEditorList 
{
    private static GUIContent
        moveButtonContent = new GUIContent("\u21b4", "move down"),
        duplicateButtonContent = new GUIContent("+", "duplicate"),
        deleteButtonContent = new GUIContent("-", "delete"),
        addButtonContent = new GUIContent("+", "add");

    private static readonly GUIContent addressContent = new GUIContent(
        text: "Address",
        tooltip: "Which OSC address to map");

    private static readonly GUIContent fieldContent = new GUIContent(
        text: "Value",
        tooltip: "Which value to bind the OSC data to");

    private static GUILayoutOption miniButtonWidth = GUILayout.Width(20f);

    public static void Show(SerializedProperty mappingList, string[] fieldList, EditorListOption options = EditorListOption.Default)
    {
        if(!mappingList.isArray)
        {
            EditorGUILayout.HelpBox($"{mappingList.name} is neither an array nor a list!", MessageType.Error);
            return;
        }

        if(fieldList.Length == 0)
        {
            EditorGUILayout.HelpBox("No fields available", MessageType.Error);
            return;
        }

        bool showListLabel = (options & EditorListOption.ListLabel) != 0;
        bool showListSize = (options & EditorListOption.ListSize) != 0;

        if(showListLabel)
        {
            EditorGUILayout.PropertyField(mappingList);
            EditorGUI.indentLevel++;
        }
        if(!showListLabel || mappingList.isExpanded)
        {
            SerializedProperty size = mappingList.FindPropertyRelative("Array.size");
            if (showListSize)
                EditorGUILayout.PropertyField(size);

            if(size.hasMultipleDifferentValues)
            {
                EditorGUILayout.HelpBox("Not showing lists with different sizes.", MessageType.Info);
            }
            else
            {
                ShowElements(mappingList, fieldList, options);
            }
        }

        if (showListLabel)
        {
            EditorGUI.indentLevel--;
        }
    }


    private static void ShowElements(SerializedProperty mappingList, string[] availableFields, EditorListOption options)
    {
        bool showElementLabels = (options & EditorListOption.ElementLabels) != 0;
        bool showButtons = (options & EditorListOption.Buttons) != 0;

        for (int i = 0; i < mappingList.arraySize; i++)
        {
            if(showButtons)
            {
                EditorGUILayout.BeginHorizontal();
            }

            if(showElementLabels)
            {
                var mapping = mappingList.GetArrayElementAtIndex(i);
                EditorGUILayout.PropertyField(mapping.FindPropertyRelative("Address"), addressContent);

                var fieldProperty = mapping.FindPropertyRelative("Field");

                var previousField = GetFieldIndex(fieldProperty, availableFields);
                int currentField = EditorGUILayout.Popup(previousField, availableFields);
                fieldProperty.stringValue = availableFields[currentField];
            }
            else
            {
                var mapping = mappingList.GetArrayElementAtIndex(i);
                EditorGUILayout.PropertyField(mapping.FindPropertyRelative("Address"), GUIContent.none);

                var fieldProperty = mapping.FindPropertyRelative("Variable");

                var previousField = GetFieldIndex(fieldProperty, availableFields);
                int currentField = EditorGUILayout.Popup(previousField, availableFields);
                fieldProperty.stringValue = availableFields[currentField];

                EditorGUILayout.PropertyField(mapping.FindPropertyRelative("Type"), GUIContent.none, GUILayout.MaxWidth(100f));
            }
            if(showButtons)
            {
                ShowButtons(mappingList , i);
                EditorGUILayout.EndHorizontal();
            }
        }
        if(showButtons && mappingList.arraySize == 0 && GUILayout.Button(addButtonContent, EditorStyles.miniButton))
        {
            mappingList.arraySize += 1;
        }
    }

    private static int GetFieldIndex(SerializedProperty fieldProperty, string[] availableFields)
    {
        var fieldIndex = 0;
        foreach(var f in availableFields)
        {
            if (fieldProperty.stringValue == f)
                break;
            fieldIndex++;
        }
        return fieldIndex >= availableFields.Length ? 0 : fieldIndex;
    }

    private static void ShowButtons(SerializedProperty list, int index)
    {
        if (GUILayout.Button(moveButtonContent, EditorStyles.miniButtonLeft, miniButtonWidth))
        {
            list.MoveArrayElement(index, index + 1);

        }
        if (GUILayout.Button(duplicateButtonContent, EditorStyles.miniButtonMid, miniButtonWidth))
        {
            list.InsertArrayElementAtIndex(index);
        }
        if (GUILayout.Button(deleteButtonContent, EditorStyles.miniButtonRight, miniButtonWidth))
        {
            int oldSize = list.arraySize;
            list.DeleteArrayElementAtIndex(index);
            if(list.arraySize == oldSize)
            {
                list.DeleteArrayElementAtIndex(index);
            }
        }
    }
}

This lets us set up our custom editor

[CustomEditor(typeof(OscMapper))]
public class OscMapperEditor: Editor
{
    private OscMapper oscMapper;

    private SerializedProperty
        oscConfigProperty,
        hostAddressProperty,
        outgoingPortProperty,
        incomingPortProperty,
        loadConfigProperty,
        mappedObjectProperty,
        mappingsProperty;

    private readonly GUIContent hostAddressContent = new GUIContent(
        text: "OSC Host Address",
        tooltip: "The IP address of the host to connect to");

    private readonly GUIContent outgoingPortContent = new GUIContent(
        text: "Outgoing Port",
        tooltip: "The port over which to send OSC data");

    private readonly GUIContent incomingPortContent = new GUIContent(
        text: "Incoming Port",
        tooltip: "The port over which to receive OSC data");

    private readonly GUIContent loadConfigContent = new GUIContent(
        text: "Load OSC config from file",
        tooltip: "Whether or not to load and overwrite connection data from oscConfig.json");

    private readonly GUIContent mappedObjectContent = new GUIContent(
        text: "Config",
        tooltip: "The configuration file in use");

    private GUIStyle headerStyle = new GUIStyle();

    private void OnEnable()
    {
        oscMapper = (OscMapper)target;

        oscConfigProperty = serializedObject.FindProperty("OscConfig");

        hostAddressProperty = oscConfigProperty.FindPropertyRelative("OscHostAddress");
        outgoingPortProperty = oscConfigProperty.FindPropertyRelative("OutgoingPort");
        incomingPortProperty = oscConfigProperty.FindPropertyRelative("IncomingPort");
        loadConfigProperty = oscConfigProperty.FindPropertyRelative("LoadConfigFromFile");

        mappedObjectProperty = serializedObject.FindProperty("ObjectToMap");
        mappingsProperty = serializedObject.FindProperty("Mappings");
    }

    public override void OnInspectorGUI()
    {
        serializedObject.Update();

        GUILayout.Space(20);
        EditorGUILayout.LabelField("Connection", EditorStyles.boldLabel);

        EditorGUILayout.PropertyField(hostAddressProperty, hostAddressContent);
        EditorGUILayout.PropertyField(outgoingPortProperty, outgoingPortContent);
        EditorGUILayout.PropertyField(incomingPortProperty, incomingPortContent);
        EditorGUILayout.PropertyField(loadConfigProperty, loadConfigContent);

        GUILayout.Space(20);

        EditorGUILayout.LabelField("Mapping", EditorStyles.boldLabel);

        EditorGUILayout.PropertyField(mappedObjectProperty, mappedObjectContent);

        if(GUILayout.Button("Refresh Available Mappings"))
        {
            oscMapper.FetchMappableVariables();
        }

        if(oscMapper.MappableVariables != null)
        {
            var mappingTargetNames = oscMapper.MappableVariables.Select(m => m.name).ToArray();
            MappingEditorList.Show(mappingsProperty, mappingTargetNames, EditorListOption.ListLabel | EditorListOption.Buttons);
        }
        else
        {
            EditorGUILayout.HelpBox("Mapping is out of sync with config. Try refreshing available mappings", MessageType.Warning);
        }

        serializedObject.ApplyModifiedProperties();
    }
}

Notice that we’re sending the potentially mappable variables through to the mappings list by extracting the names from the Mappings property of the OscMapper.

Example of mapping

Mapping for a project I was working on.

Corresponding layout in TouchOSC Editor.

[CreateAssetMenu(fileName = "Config", menuName = "Config/NeverCloserConfig", order = 1)]
public class NeverCloserConfig : ScriptableObject
{
    [OscMappable]
    public float ClosestDistanceWeight  = 1f; 
    [OscMappable]
    public float TimeInFrustumWeight  = 1f;

    public Vector3 DefaultStartCameraPos  = Vector3.zero;
    public Vector3 DefaultInputScaling  = new Vector3(1.0f, 0.8f, 0);
    public Vector3 DefaultProjectionWindowOffset  = Vector3.zero;
    [OscMappable]
    public float DefaultProjectionWindowRotation = 0;

    #region Osc Mapped properties
    [HideInInspector]
    [OscMappable]
    public Vector2 StartCameraPosXY
    {
        get => new Vector2(DefaultStartCameraPos.y, DefaultStartCameraPos.x);
        set
        {
            DefaultStartCameraPos.x = value.y;
            DefaultStartCameraPos.y = value.x;
        }
    }

    [HideInInspector]
    [OscMappable]
    public float StartCameraPosZ
    {
        get => DefaultStartCameraPos.z;
        set => DefaultStartCameraPos.z = value;
    }

    [HideInInspector]
    [OscMappable]
    public Vector2 InputScalingXY
    {
        get => new Vector2(DefaultKinectInputScaling.y, DefaultKinectInputScaling.x);
        set
        {
            DefaultKinectInputScaling.x = value.y;
            DefaultKinectInputScaling.y = value.x;
        }
    }

    [HideInInspector]
    [OscMappable]
    public float InputScalingZ
    {
        get => DefaultKinectInputScaling.z;
        set => DefaultKinectInputScaling.z = value;
    }

    [HideInInspector]
    [OscMappable]
    public Vector2 ProjectionScreenPositionXZ
    {
        get => new Vector2(DefaultProjectionWindowOffset.z, DefaultProjectionWindowOffset.x);
        set
        {
            DefaultProjectionWindowOffset.x = value.y;
            DefaultProjectionWindowOffset.y = value.x;
        }
    }

    [HideInInspector]
    [OscMappable]
    public float ProjectionScreenPositionY
    {
        get => DefaultProjectionWindowOffset.y;
        set => DefaultProjectionWindowOffset.y = value;
    }

    #endregion
}

Here you can see that I’ve mapped controls with the addresses corresponding to the ones in unity. Note: I haven’t found a good 3D control for mapping Vector3 hence the properties that expose the Vector2 + float versions of the Vector3 fields. Also note that touchOSC sends the XY-pad control data with in the order of Y, X.

Caveats

The component currently only receives data. It’s fairly easy to set up sending of data as well as receiving, You can send data by calling OSCHandler.Instance.SendMessageToClient(TRANSMITTER_NAME, “/Some/Address”, ObjectToMap.SomeVariable); in OscMapper.

At the time of writing the OSC listener is only started when the application is running (i.e. not in editor mode), but it could potentially work in editor mode as well, which is arguably the most useful use case when working with ScriptableObjects.

When creating the layout for TouchOSC you have to set the range for the values in the TouchOSC Editor or whatever OSC controller you end up using.

Future work

This component can certainly be improved upon. Here are a few ideas to follow up:

• Create two-way mapping to variables so they can be sent as well as received automatically. Possibly SendOnly and ReceiveOnly flags as well.
• Custom names for variables via the OscMappable attribute
• Add Scalars for the mappings so that all controller layouts can use normalized values. This makes it clearer what the ranges of each value can be. (Could potentially be set by a custom OscRange attribute)
• Create an export/import format for mappings, so mappings can be added to a built application.
• Add mapping to functions (useful for buttons in the controller layout)
• Optimization — I’m sure there lots of room for optimization here 😅

Thanks for reading 😃