Unity UIElements: the first impression in production
What am I going to share with you today is an interesting framework from Unity — UIToolkit, also known as UIElements. We are going to cover its main features without diving deep into the code. It is worth clarifying that the team members have no previous experience of working with web layout and things that are obvious to professionals in this area may not be so to us.
Background
Unity originally had an IMGUI system that was built entirely from code. Such a system was inconvenient for at least two reasons: a lot of code and the resultseen only after compilation at runtime. IMGUI was replaced by the UGUI system, which we have been using. The new system contains main components for working with the UI (Canvas, Image, Button, etc.). Although the new systemis much more convenient than IMGUI, there still were some problems: the complexity of the layout (including adaptive), many elements in the hierarchy, a complex architecture, slow work, etc.
Understanding the problems of UGUI, developers have created extensions for the Unity Asset Store. For example, node systems like DoozyUI, or a simple feature plugin that could work ut-of-the-box — rounding the edges of images like Procedural UI Image.
Concept
A system UIElements (as known as UIToolkit, but you’re more likely to come across the first name) comes out with Unity 2019.1 and it introduces a new user interface structure for Unity. It is important to note that the emphasis in the framework is put on the extension for the Editor. Moreover, runtime support is being implemented consistently and is going to be included in the roadmap.
An idea of interface building system has been borrowed from a well-known web layout based on HTML / XAML / XML and CSS files. UIElements has formats’ analogs: UXML is used to build a hierarchy of objects whereas USS is used to create style classes, i.e. everything that is responsible for the size, position and shape of the UI element. When it comes to structure of the format, there is absolutely no difference in comparison to the original formats, except a slightly simplified syntax and limited functionality.
UIElements’s main objectives
Our plan was ambitious: we need to use the preview package in production. A question arises: what made us think about this?
We have quite a large project that has been developed for several years and it is a large corpus of data visualization, and, in addition to 3D, it contains a bunch of similar and diverse 2D content at the same time, represented mostly by graphs and diagrams in different configurations. Thus, you can imagine the number of prefab options, the degree of object nesting and the complexity of the GUI system. The UGUI familiar to Unity relies on components such as Canvas, CanvasGroup, LayoutGroup that are not always convenient and often show poor performance. Adding and removing elements cause an implicit rebuild of canvases, and the layout is not formalized, which is considered to be not acceptable.
As a result, a pool of tasks has been formed that should be solved by UIElements.
First of all, working at runtime. Indeed, the package was originally created to work in the editor, and in this case it somehow copes with this task. However, the announcement of runtime support got quite a lot of attention at the Unite conference in 2018.
Secondly, the creation of reusable and extensible UI components. The number of elements and their variations have grown significantly in the project, and it is becoming more difficult to manage them in context in the UGUI system.
Thirdly, reducing the load of 2D GUI on the system. Who wouldn’t want to optimize the UI and reduce the number of draw calls? So did we.
Fourthly, approaching the creation of a UI constructor. The project is moving in a direction with more distinctive autonomy, the user should be able to construct the desired set of data and its presentation, in which case the layout should be flawless and strictly formalized. This can be done with UGUI, but considerable effort is required.
On the one hand, such projects in Unity are quite rare, but on the other hand, the problems are actually familiar to everyone who participated in the Unity development of projects with a rich graphical interface. Convenience and performance are definitely the weak points of UGUI.
Our experience
We spent more than one sprint on learning and integrating UIElements in a production project. We are using the 2019.4 Unity version, and have not tested the new framework features from the 2020.1 update.
Runtime capabilities. In order to understand whether this framework can be used at runtime, you need to test all the necessary features on the target platform. Due to the fact that the framework was originally developed to extend the editor, you need to keep track of the differences in runtime capabilities. In our case, all tests relate to this very aspect.
Entry threshold. It is high. As an approach UIElements differs significantly from UGUI. It should be a Unity developer who will spend enough time to delve into the system for creating UI controls and who will break through implicit documentation and the forum. You can’t just sit down and start working after watching one tutorial and looking through the documentation. To put it mildly, many developers in the team were dissatisfied with the ratio of UIElements’ production readiness and the entry threshold for comfortable work.
Flexbox. I understand that most people familiar with web-layout will not have any problems creating a responsive panel using flexboxes, but the average Unity developer cannot do this, because he usually does not need it. As you can see, USS is still different from CSS and this must be taken into account when coding and layouting. Even an experienced specialist needs to understand the specifics of UIElements. However, when you understand how it works, layout becomes much easier, and the UIBuilder as a working panel is a pretty convenient tool for layouting setting style. Layout on UIElements is still more convenient than layout on UGUI, because there are simply no flexboxes on UGUI. However, it requires knowledge and skills in completely different areas. Many will notice that layout is not a programmer’s job. I will talk about this a little bit later.
Materials and best practices. It doesn’t exist. Where can you find useful information? In the documentation? It is superficial and raw. In articles and tutorials? There are only a few official tutorials without deep diving, there are no other sources. On the forum? Yes, they share specific problems there, and you need to find your problem and hope that there is a solution. But let’s be honest: lack of information is actually quite common. I regularly meet situations when a well-known technology or tool does not incorporate all edge cases and all best practices. There is always a process, always a research, so the argument is rather controversial.
A variety of ways to create and use elements. You can use different degrees of UIElements involvement in the UI logic, from creating components entirely through the code in the best traditions of IMGUI to presenting a “view”as passive as possible, and working directly with the layout through the code. It seems to me that the most interesting way would be an intermediate option, when we have a certain component that implements all the specifics of the presentation through the UIElements system and works with a higher level through the usual more abstract methods — this will allow, if desired, relatively painlessly substituting implementations and doesn’t depend on UIElements on higher level. Although, of course, the approaches may be different.
Styles workflow. This is a big plus, of course. Working with styles was quite an enjoyable experience after UGUI, even if everything was not immediately clear. However, we faced a number of problems and pecularities here. Our design is being developed in Figma, and we were absolutely ready to set up parsing in Unity, but it turned out to be not so easy. Not only does Figma work poorly with flexbox, but also the desire to parse CSS imposes certain obligations on the layout of the design and on the hierarchy structure, which is no longer dependent on the developer and is poorly controlled. At the same time, USS differs from CSS in a more limited set of functions, and this should also be taken into account when layouting, otherwise there will be errors. On this step we have not started working in Unity and UIBuilder yet. UIElements have inlined styles, which means that the element’s style information is directly contained in UXML. This kind of an existing feature is in fact extremely undesirable and not recommended for use, at least because it is not supported in the build. It is an easy thing to miss, especially if one does not have enough experience. Moreover, in case of using styles with the same custom properties on a single element, you can get a style conflict, which you will notice after getting a random result. Nevertheless, many of us were satisfied working with styles precisely for the reason that it brings more clarity and convenience than UGUI.
Animation possibilities. In fact, they hardly exist. We used DOTween Pro to create the animation, but it’s necessary to keep in mind that the project uses fairly simple 2D animation, and the needs are almost completely covered by DOTween. With the update in the 2020.1 version, its own animation appeared, however, it is quite simple in functions and generally simpler than DOTween, although it is most likely that the developers were initially inspired by it. Here are some examples from UIToolkitUnityRoyaleRuntimeDemo:
Listview in UIElements is a very useful tool that allows you to keep hundreds of clickable objects in a list without sacrificing performance. There is no such functionality out-of-the-box in UGUI, but there are quite good assets like EnhancedScroller that implements it. And as it usually happens, this tool is not free.
However, we found one interesting feature in the Listview usage. Its functionality turned out to be not completed. For example, the EventSystem can use two types of Event Generation as an input system: Read Input as a default and IMGUI Events.
With the Read Input option, the clickability of the Listview will not work because the ProcessMouseEvents () method of the UIElementsEventSystem class does not process or send information about the number of clicks. When you switch to IMGUI Events mode, the list starts working. An interesting detail that you might find useful. You can work on this part yourself, but let’s leave that to the guys from Unity Technologies.
The framework parsing system is currently experiencing certain difficulties. As a result it can lead to different outcomes and issues. For example, an error occurs while writing and reading CSS properties that are default for a component. More details can be found HERE. Also custom properties of the component inherited from VisualElement are displayed incorrectly in the inspector, which complicates the work with UIBuilder. Or, for example, the impossibility of working with inlined styles which, on the one hand, exist, and on the other, do not work in any way. And if they do work in general, they definitely don’t in the build. At least for now. More details can be found HERE. However, work on UIElements is very intensive and many bugs are being fixed.
Layout specialists. It is difficult to find them, and in fact it is impossible, because no one works with such a technology stack. Keep in mind the possibility that programmers will layout and complain, or teach a layout designer to work in Unity and the specifics of UIElements.
Cross-platform, which is what Unity is often chosen for. Unfortunately, a full-fledged cross-platform is not yet implemented. The project being tested on Windows is planned to be ported to WebGL, but this particular platform does not work, distorting the visual in the build. Because of this, we had to abandon the use of UIElements in this project. The problem is known and they plan to work with it, details can be found in THIS forum thread. Therefore, of course, it is necessary to familiarize yourself with the documentation and the forum as much as possible for potential problems on the target platform.
Conclusion
In conclusion, I would like to say that the experience with UIElements was really interesting, because you need to address questions that Google often cannot, the documentation doesn’t always contain the necessary details, and the Unity forum on the topic has dozens of threads created by people struggling with this issue.
Do you need to learn and try to implement this framework, despite all the flaws that were mentioned above?
We came to the conclusion that at the moment UIElements can be used only in a limited type of projects on mobile and PC platforms, we plan to test its production application on small projects, the technical requirements of which meet the current capabilities of UIElements. The developers “subtly” hint that it will be developed in the future, rather than UGUI.
