The Roundup of How to Create a Massive Simulation Environment for Intelligent Transportation? |51TECH
51TECH is 51WORLD’s brand-new column, delivering the frontline analysis of cutting-edge and hard-core technologies.
We believe that technology transforms our life, we believe in the goodness of technology, and we hope to share our experience and thinking with the industry at 51TECH
In recent years, the concept of “big world” put forward in the field of smart transportation has posed new challenges for related simulation. Among them, the four major challenges for the static simulation of intelligent traffic are the construction of super-large maps of hundreds of square kilometers at the macro level, the integration of digital assets in the “big world” at the meso-level, the optimization of rendering efficiency on multiple platforms, and the construction of a composite digital asset library at the micro-level.
In this issue of 51TECH, we invite Wang Chao, the content director of 51WORLD Intelligent Driving and Transportation Division, to talk about the upgrade of static environment simulation technologies and the detailed explanation of creating a large traffic simulation environment.
The content team of 51WORLD Intelligent Driving and Transportation Division has been committed to providing full-stack content solutions for autonomous driving simulation and traffic simulation since its inception. It has accumulated rich experience in the rapid production of simulated maps, automated map generation, efficient rendering on multiple platforms, and aesthetic balance after delivering nearly 100 projects.
As a seasoned frontline static simulation engineer, Wang feels the challenges facing the current industry.
Part.1 Constructing and balancing the “big virtual world”
As the advent of the 5G era has pushed intelligent transportation to a new height, the concept of “big world” came into being in the field of traffic simulation. The big world denotes the upsurge of demand in the range of simulated visual maps and the length of road network. A transportation project may require hundreds or even thousands of kilometers of large or super large simulation maps for training, collecting vehicle data set, and assisting in decision-making. It also means an exponential increase in the volume of simulation functions.
Four major challenges come on the heels of this burgeoning industry: the construction of super-large maps of hundreds of square kilometers, the integration of digital assets in the “big world”, the optimization of rendering efficiency on multiple platforms, and the construction of a composite digital asset library.
01 Construction of super-large road networks
Since the localized simulation maps of the first perspective are limited and inadequate for smart transportation, the free-view maps with the car-tracking perspective and the bird’s-eye view are constantly yearned for. However, it needs an accurate restoration of hundreds of kilometers or even thousands of kilometers of the simulated terrain to realize a bird’s-eye view from the third perspective. For example: to simulate a 100- square-kilometer scene, it is necessary to recreate roads with a total length of 300 kilometers, of which about 40 kilometers are highways or first-level roads, about 160 kilometers are urban expressways or second-level roads and 100 kilometers of branch roads or roads of the third level and below, thus forming a huge road network.
Therefore, we need to efficiently integrate and utilize limited computing resources, so as to complete such large-scale simulation maps. Next, I will explain how we significantly reduce the unreasonable resource consumption of the GPU with stress tests.
How to do stress testing?
It must be clarified that since a traffic simulation map is the mapping and restoration of the real world, it depends on various data sources, including point cloud data, tilt photography, satellite images, panoramic images, aerial video, etc. However, these real data are more or less problematic, so a secondary design is needed to achieve accurate geographic location, matching regional style, and smooth operation efficiency.
We conduct research with our accumulated blank model testing data to evaluate the integration of the road network and the surface, and calculate a set of map segmentation schemes to facilitate the program to efficiently load each atomic map, thereby enabling transportation functions to make more flexible. In addition, we subdivide the smooth meshes to no less than 3 times, simulate no less than 100 dynamic vehicles and no less than 100 moving pedestrians, and adapt complex coloring and high-precision textures to the components of the blank model for specific tests. At the same time, we add greenery of no less than 5000 types to verify the frame rate of static scenes. 。
We group the testing according to the types of rendering, such as road networks, buildings, vegetation, etc., and strictly control the budget after stress testing, in order to improve the team’s efficiency, thereby shortening the development cycle. It is worth noting that even without ForceCulling, LOD, and HLOD, the current stress test method we use is generally very efficient, because this method can quickly assess the impact of the asset on the map rendering and quickly calculate the budget for various types of resources such as buildings, roads, surfaces, vehicles, and pedestrians, so as to reserve more space for dynamic functions.
When the stress testing is done abundantly, we can optimize the product for different situations. Next, I will talk about the difficulties in restoring the road network.
The road network should reveal the complexity of the real road, such as the undulating conditions of the road surface, the changes of the intersections, etc. Meanwhile, it should be able to meet different needs of smart transportation such as parallel computing, sensor simulation, vehicle dynamics, etc. Therefore, the production of a traffic simulation road network cannot be done simply with the curve editing tool of Spline.
51WORLD independently developed the WorldEditor to automate the geometric data and UV mapping of a road network, thus greatly improving the efficiency of producing a road network, on which a whole simulated world can be developed.
At present, our team is capable of producing large maps of 100 square kilometers, and we aspire to rapidly recreate larger maps of several square kilometers and construct the road network of thousands of kilometers. Next, I will talk about how to effectively integrate digital assets in producing maps.
02 Integrating digital assets in the “Big World”
The Big World is a combination of small cell maps, each containing a huge number of digital assets. Different projects have different acceptance criteria. Now, we integrate digital assets mainly in two directions: one is to add semantics, and the other is to lighten heavy assets.
Adding semantics to digital assets: assets used in sensor simulation are typical heavyweight digital assets and of the highest level. As time and weather vary, semantic information should be added to meet different requirements of sensor recognition training, such as the parameter of wet and dry environment changes, snow stains coverage parameters, etc. Up till now, we have processed more than 40 semantic heavyweight digital asset types, which maximizes the restoration of object properties and material changes in the real world.
The lightening of heavy assets is often used for the rapid retrieval and use of assets. For example, the digital assets loaded in the WorldEditor need to be lightweight. At the appearance, it seems feasible to make multiple versions of assets to meet different needs. However, the extremely high costs put off even the most fervent tech-lovers.
Therefore, we define varied requirements in different levels of LOD, encapsulate these assets with BP, upgrade and downgrade them more flexibly, adapt appendages, and replace materials to support sensor simulation, case testing, and vehicle dynamics.
For example, we divide the attributes of real-world vehicles into static and dynamic. In the static dimension, lightweight assets need a reasonable structure, a low number of faces, and a small number of materials, while heavyweight assets need to be clearly structured with rich materials and realistic textures. Meanwhile, both of them can flexibly add appendages, such as bus license plates, advertising stickers, warning lamps, etc. In the dynamic dimension, the first consideration should be given to adapt corresponding skeleton animations to different levels, restrict the luminous intensity, and control the number of particles, etc. Therefore, flexible upgrades, downgrades as well as adaption become the kernel of making composite assets.
03 Establishing a composite digital asset library
A digital asset library is the smallest unit to build a map. Whether a single asset meets the rendering standards in the field of traffic products directly affects the performance of a simulated map, which determines the feasibility of constructing a big world. Therefore, it is instrumental to build a composite digital asset library. We assess digital assets in terms of aesthetics, compliance, and feasibility.
Aesthetics: we should evaluate whether a digital asset faithfully restores the properties of objects in the real world, such as surface materials, size ratio, etc. and whether a digital asset meets the public aesthetic standards. For example, we construct pedestrian characters with 3D footage from live scanning and create behavioral actions from action sequences generated by Motion Capture, to make them more credible and beautiful.
Compliance: The compliance of a digital asset will have a direct impact on the cost of its subsequent iteration. The assessment of compliance not only includes models and materials but also hidden functions such as vertex shading, the role of multiple sets of UV, model LOD, physical materials collision occludes, luminous parts, multi-dimensional sub-materials, bones, audio code rate, animation frame rate, particle size range, source file format, naming rules, storage location, etc.
Feasibility: When a digital asset is referenced by a map, first it must be flexibly used by the level designer. And the second modification will not lead to repeated revision and construction of the level. Last, it can be modularly combined into a new resource. The gist lies in whether an asset has been dismantled carefully in the early stages of production.
In static environment simulation for intelligent transportation and autonomous driving, the closest visual distance of a digital asset should be less than 1m, which is highly dependent on the digital asset library. The asset should be able to be displayed on multiple platforms, such as PC-based VR, MR and XR, and the access of cloud rendering also needs to consider the size of the resource package. It emphasizes the functions and effects, but the digital asset needs to be lightened, the seemingly contradictory requirement of which also exists in the film and game industry.
Our digital asset library consists of nine categories that assume the largest elements of the real world. Each category is divided into no less than 8 sub-levels. In principle, any static digital asset should be dynamically scalable. Therefore, we’ve reserved the functional interfaces, such as LOD, UV, materials, and vertex information for a static 3D model, so that engineers can freely combine the tools into dynamic elements available, which are, of course, part of the asset library.
Since simulating each 3D asset in the virtual environment takes a large amount of time, it is unattainable to process every digital asset to be extremely aesthetical. If we grade the digital assets of film and games at 10, these required by the digital twin can only get 6. It also seems realistic to remove redundant artistic details and return the models, textures, and materials to the original state.
Although we don’t agree with each other in aesthetics, I believe that digital twins must meet the basic standards, such as the blue sky on a sunny day, reflections on the surface of the water, etc. How to strike a balance between intuition and emotion is the basis for defining digital assets, which is also bound up with costs.
04 Rendering efficiency on multiple platforms
Due to the secondary design of the maps, performance stress testing, and the maintenance of standardized composite digital assets, the rendering efficiency of transportation simulation projects we delivered in multiple platforms is generally up to standards, even outperforming peer products. At present, intelligent transportation and autonomous driving simulation projects are mostly based on the PC platform. Against this backdrop, how to realize efficient rendering on other platforms such as 5G cloud rendering streaming, VR and MR poses another challenge for the industry.
In addition to the scale of the map, the digital asset itself and the toolchain also affect rendering efficiency. Just like running, being able to run is one thing, but to run fast is another. It is not easy to run with data and functions. Of digital assets, different perspectives, models, textures, materials, lights, motion sequences, special effects, sound effects, UI, etc. also have an impact on rendering efficiency.
How to improve the rendering efficiency of the simulated maps? It is ideal to work with lightened maps on high-performance PCs.
Currently, we have achieved the following goals in traffic projects based on PC hardware:
[Successful implementation cases]
Up to now, our team has guaranteed the frame rate in the L4 static simulation maps of hundreds of square kilometers (51WORLD all-element scenes are divided into five levels from L1 to L5, with the scenes of L4-L5 being able to restore detailed elements such as roads, traffic facilities, vehicles, and pedestrians), and it has been verified on recent large projects.
