Using Unity for a 3D Simulation of Crowds in 5G Networks

The research project “roverNet“ of the University of Applied Sciences Munich actively researches intelligent connected mobility in 5G networks. The aim of this project is therefore to systematically investigate the interaction between mobility behaviour and wireless networking using characteristic scenarios and to develop suitable methods for robust networking and information dissemination in these mobility scenarios. While microscopic simulations for crowds and networks are common, they are considered and evaluated independently.

Since traffic participants and the network infrastructure influence each other, as in the network outage of the Boston Marathon bombing that was caused by spikes in usage, it is required to test them together. The “crowNet“ project couples state-of-the-art microscopic pedestrian and crowd simulation frameworks with 5G network simulations.

Microscopic Locomotion Model

Microscopic models simulate the motion behaviour of each individual person and car differently than macroscopic models where crowds are globally simulated. Over time, a variety of mathematical models have been established to simulate movement behaviour. Regardless of the underlying mathematical approach, each model follows the same approach.

Due to the complexity of locomotion models multiple frameworks have evolved to support creating, evaluation and running simulations like SUMO and Vadere.

OMNeT++

Since locomotion models are coupled with 5G network communication OMNeT++, an extensible and component-based C++ simulation library for building network simulations is used as an overlying simulation engine. OMNeT++ itself is highly object-oriented and therefore supports inheritance and interfaces with modules subject to a hierarchy. Extensions developed and maintained by other companies or researchers offer functionality such as Simu5G, that is used for 5G network communication. Vadere and Sumo connect to OMNeT++ via TRACi, a TCP server/client-based interface for retrieval and manipulation of simulation objects.

Data Access

OMNeT++ does not support passing internal data outside of the box. Due to the object-oriented approach, every feature has its own C++ implementation file that holds for Unity necessary information such as position data and a unique identifier. This applies to people, cars, and the antenna. OMNeT++ itself uses Signals to pass around class pointers. Signals are based on the sender and receiver template, where data is written to a named endpoint and is accessed by one or multiple listeners when new data is present.

Data Forwarding

Received class pointers are then cast back into their respective implementation giving access to the data, but in addition showing which action triggered the signal. The movement of a person in Vadere is cast to a different class than the person’s movement in SUMO. This is not only bound to the movement but also the network information provided by Simu5G. Therefore all messages are accessible globally inside of OMNeT++. All this information is collected by a new module and sent over a TCP socket. The following information is sent for each action:

• ObjectType (class sending the signal: car, vehicle, antenna, packet)
• SourceID (Unique ID of the caller)
• TargetID (only set if the ObjectType is a packet)
• X -, Y- and Z – Coordinates

This information is stored in the JSON file format and used globally in every software project to support a generic software architecture. When required it is enabled easily inside the startup configuration, where in addition the target IP and PORT are defined.

Unity

Unity is a cross-platform game engine widely used in the fields of 2D, 3D, VR & AR applications. The graphical user interface in combination with C# allows fast prototyping and development. Creators share scripts, models, and utilities in the asset store.

Environment

The conversion from the 2D OMNeT++ representation also requires the environment to add the third dimension. Hand build environments of detailed cities would require lots of time and therefore a 3rd-party tool is used.

CityGen3D allows fast rendering and the generation of vegetation, buildings, and streets based on longitude and latitude coordinates. The scene itself consists of multiple reusable components that are extendible and allow the user to change the generated map to his liking.

Models

The asset store of Unity has a great selection of free-to-use models for persons and vehicles. More complex ones, like the antenna, are modeled in a blender by hand and then exported to Unity. The graphical user interface allows quick changes to models even without knowledge of the codebase, allowing adoptions and maintaining the generic implementation.

Process

The entire Unity process depends on the JSON message format used throughout the project. Via the object type, each message is assigned to either a person, a vehicle, the antenna, or a packet. If the SourceID is already present, the object is moved to the new location or, in the case of a packet, displayed with an arrow. The TargetID is only set if it is a packet, as a receiver is required. Depending on the sender, the color of the arrow is adjusted to make clear who is the sender and who is the receiver. This generic approach enables future additions of entities, such as motorcycle drivers, where only a suitable 3D model is required and it has to be added to the ObjectType pool.

Message recording and playback

A huge drawback of OMNeT++ and large scenarios, in particular, are the missing recording and playback options. The runtime of the Scenario is often several hours long. If a certain scene is to be viewed within this time window, this requires repeated simulation, until the required time stamp. Furthermore, it is not possible to rewind or set the playback speed dynamically. Although OMNeT++ provides a way to store messages from signals in a database, this is inconvenient and does not allow access at runtime, making it rather impractical for testing. Instead of sending the messages directly to Unity a new module is added, receiving messages from OMNeT++ and then potentially forwarding them to Unity. This enables live data manipulation, extensive logging, and recording to an external database.

ECAL

The event controller and logger (ECAL) is both a client, receiving messages from OMNeT++ and a server forwarding them to Unity. The C# module is deployed inside the same docker network as the OMNet++ project and therefore reduces the network overhead. Messages can be manipulated in real-time, postponed, or not sent at all. This allows precise adjustments to the representation in Unity based on the requirements. In one run, cars maybe should appear bigger in size or only half of them. This is all handled inside ECAL as Unity is only used for display and the concerns are separated.

InfluxDB

The database choice for this requirement is InfluxDB, that allows precise timestamps guaranteeing the integrity of the scenario, where messages are only 2µs seconds apart from each other. The feature-rich graphical user interface allows live viewing of the data and executing queries with the help of the query builder. In addition, stored messages are easily shared, allowing playback on the same device where Unity is running.

Playback

The recorded scenarios are accessed by a Python script that accesses the InfluxDB and sets parameters for ECAL to send messages to Unity. This allows the adjustment of the start and end time of the data stream of InfluxDB and playback speed by changing the pause time between messages. The playback of a scenario therefore only requires the Unity instance, ECAL, and InfluxDB, all available in a single docker project, greatly reducing the setup- and execution complexity of the project.

Evaluation

In an example scenario covering 50 persons, 46 vehicles, and 1 antenna simulated over 10 minutes in real-time the simulation reached 121 seconds storing a total of 1.3 million messages with a 6 GB database size. Roughly 95 % of these messages are packet transmissions, i.e. 5G messages, whereby 80 % of these messages are sent by a person either to the antenna or another person.

Conclusion

Exporting data from OMNeT++ is not only possible without any major performance restrictions, but offers great functionality in combination with the use of an external database. The strict separation of concerns allows accelerated development and adjustments without knowledge of other software dependencies that are not required to be changed. Unity therefore enables the abstract research of intelligent connected mobility to a wider audience, without limiting the existing implementations.