Time Sensitive Networks — Audio Video Bridging
Have you ever noticed how the live streaming of an event becomes uninteresting when the video and audio of the event are played on big screens without synchronization? It is generally termed as lack of lip sync.
Audio Video Bridging (AVB) is a technology developed to address the streaming issues in industrial events, where live streaming events faced issues due to loss of synchronization between audio and video data, played at different points. AVB is presented as a protocol enhancement to the Ethernet network, which is the cheapest and most commonly available networking technology available.
AVB in automotive network:
Over the past few years, multimedia technologies used in automobiles have increased considerably, owing to the consumer demands and evolving technologies. Features such as media playback from multiple sources (USB / Bluetooth / Wi-Fi / YouTube / Radio), surround view cameras, and built in navigation support have become commonplace options in many mainstream automobiles. Rear Seat Entertainment (RSE) units are growing in sophistication with more sources and choices at your fingertips. Each of these options has added to the need and desire for a common networking architecture in the automobile.
Even though CAN is the universally accepted networking technology in automobiles, the varied requirements of multimedia usage like scalability, QoS, cost, bandwidth requirements provides an open door to other technologies as well.
The networking solutions used in automobiles have different constraints ranging from the deterministic nature of network to the temperature sensitivity of the cabling materials used. The IEEE 802.1 Audio Video Bridging (AVB) task group and the IEEE 1722 AVTP working group offers a standards-based approach for highly reliable networked transmission for low-latency applications like those found within an automobile. Wired Ethernet networks employing AVB protocols are very well suited to automotive deployment due to both simplified cabling and reliability of a hard-wired solution.
Initially, AVB was used in automotive networks for infotainment systems to distribute media data and also for transmitting control data among other components in the cockpit domain.
How AVB works
AVB is an enhancement to the Ethernet suite of open standards. It provides quality of service (QoS) guarantees, a network time synchronization service, and a related transport protocol for transmission of time-sensitive traffic that together allow a network to handle audio-visual (AV) data. In the automotive environment, it can also satisfy more generalized time-sensitive networking requirements, opening up the possibility of a single network that handles infotainment, body control, driver assistance and even safety-critical functions.
AVB uses different concepts to meet its requirements, which are defined in different Ethernet standards as amendments. The major concepts are network time synchronization, data priority, stream reservation and traffic shaping and queueing. To enhance the feature sets, there is dynamic device discovery and enumeration and control features, but this is rarely used in automobiles.
AVB network consists of Talker — the component that streams the media (camera, media player, mobile phone), Listeners — component that renders the received media (display module, speakers) and AVB switches and bridges — connecting the different AVB and non-AVB components in the network.
An ethernet network becomes AVB enabled, when the clock of all the AVB enabled devices becomes synchronized with a Grand Master clock. The grand master clock is provided by one of the network components and all other devices will accept the master clock as their clock. Without synchronizing, AVB communication does not happen. When a talker is ready to do the streaming (a user selects play button), it sends out a Talker Ready message, with the stream information and the switch will send this packet to all the connected devices. The listeners, who are capable of rendering the talker’s stream responds with Listener Ready message. The stream is reserved by MRP packets, where the talkers and listeners join a VLAN with MRP Join message. The talker will start streaming, as soon as it receives at least one Listener Ready message. Once the streaming is completed, the devices leave the VLAN using MRP Leave message and the stream reservation ends.
All the AVB packets will contain a time, presentation time, which specifies the time on which the data should be played back. This time is based on the synchronized clock and is calculated based on the listener who is farthest from the talker. All the listeners will be able to recreate the media data and the presentation time from the received packet and will be played at the presentation time specified. This is how AVB achieves synchronized play back at all the listeners.
AVB Network Components:
AVB Network consists of AVB enabled bridge and AVB enabled end points. The network limit stops when a non-AVB bridge comes in between.
AVB Switch / Bridge — Identifies AVB messages, routes it to the right devices.
End points — Talkers and listeners.
In an AVB network, upto 7 hops (bridges) are supported. The maximum network delay between a talker and a listener in an AVB network is 2 ms. This means that even if there are 7 bridges between a talker and a listener, a packet will reach from the talker to the listener within 2 ms. This is ensured by creating and communicating the presentation time of each frame by the talker. The presentation time is calculated based on the link delay to each node in the network.
Technology Overview
The IEEE 802.1 AVB and IEEE 1722 standards form the foundation for the technology promoted by AVnu alliance, along with the enhancements to other 802.1 technologies for bridges and switches.
The foundational standards for AVB are explained in the below sections.
IEEE 802.1AS (PTP) based on IEEE 1588:
PTP is Precision Time Protocol, which ensures that all AVB capable devices in a network uses a single clock provided by a common Grand Master. There can be multiple devices in the network capable of providing master clock. The Grand master is selected from these devices by running the Best Master Clock Algorithm (BMCA). The devices with the same grand master shall be able to communicate with each other directly. The messages that are used in time synchronization mechanism are: Announce, Sync and Follow up. The devices in the network calculate the delay from each other by sending Path Delay Request and from the Path Delay Response. The synchronization message flow is:
IEEE 802.1Qat — Stream Reservation:
AVB has two classes of data — class A (every 125 micro seconds) and class B (every 250 micro seconds). The media data (audio and video) are normally sent using class B. To achieve this transmission timing requirement, the available bandwidth has to be reserved when there is an AVB data is ready to send. The bandwidth reservation shall also be teared down when the transmission is completed. A maximum of 75% of available bandwidth shall be reserved for AVB transmission. This stream reservation is defined by 802.1Qat specification. This is normally implemented by talkers and AVB switches.
When a talker is ready to transmit data, it will send a Talker Advertise with the stream information — type of stream, media type, etc. When a listener who is ready to accept and play back this stream receives the advertisement, it will respond with Listener Ready message. There can be more than one listeners sending the Listener Ready, accepting all or partial media announced by the talker. The talker will start the transmission only if it receives at least one Listener Ready message. If there is error, Talker Advertise failed / Listener Ready failed are the messages transmitted.
Before starting the transmission, the bandwidth will be reserved by sending MRP request and the talker and listeners can join a VLAN announced in the MRP using MRP Join request and can leave the network using MRP Leave request. This will also tear down the bandwidth allocation.
IEEE 802.1Qav — Forwarding and Queuing
Along with stream reservation, a forwarding and queuing mechanism is also implemented in AVB to ensure that the packets are not dropped. Like SRP, talkers and bridges implement FQTSS and this is mostly implemented in the hardware. Credit shaped algorithm is used for the queuing mechanism and the credit is given if at least one frame is available in the queue and the transmitAllowed signal is True for the queue. The packet is launched when the credit reaches a maximum value. It ensures that if a frame is queued in the priority class, it will get the transmission bandwidth at regular intervals, which will not be more than 125 micro seconds for class A and 250 micro seconds for class B.
IEEE 1722 — AVB Transmission Protocol (AVTP)
AVTP specifies the packet format and the requirements to transmit an AVB packet through network. Talkers start the transmission and listeners receives and recreates the packets. The bridges ensures that the packets are sent at proper time and stream reservation is created and removed according to the transmission .
One important point to notice in AVTP is that the packets are transmitted to pre-defined multicast addresses. Point-to-point transmission is not done in AVB. This is because there can be multiple listeners, waiting to receive the packets and all of them need to play back the data at the same time. Point-to-point transmission will need more bandwidth and the timing issues will occur.
Each AVTP packet will contain the streaming data, along with a timing information that mentions when to play back the received data. This presentation time is provided by the talker and is calculated based on the listener node that is farthest from the talker. All listeners recreate the presentation time and it will be aligned with the synchronized global time, hence the time for the packets at all listeners will be same. This is media clock recovery.
IEEE 1722.1 — Discovery, Enumeration and Control (AVDECC)
In networks, where nodes (talkers and listeners) are dynamically added and removed, device discovery protocol is used. A control module is needed to handle the AVDECC protocol, which can be a stand alone module or part of any other node.
The AVDECC protocol lets the nodes identify the stream ID and verify if it is suitable for the node configurations. It also allows control data flow — like volume control dynamically without the use of a parallel communication path. The features of a stream will be announced by the talker and it will be enumerated in a standard way so that the listeners and bridges will be able to decode and understand.
Open AVnu
OpenAvnu is an AVnu sponsored repository for Time Sensitive Network (TSN and AVB) technology. The code is in C programming language, works in Linux operating system and is primarily created for Intel’s i210 Ethernet card as the hardware platform.
The openAvnu source code is used by most of the AVB / TSN stack developers to customize their own stack since it provides a base for directory structure, build mechanisms and source code arrangement.
It supports different media mapping schemes and also provides sample applications to use the stack.
TSN (Time Sensitive Network)
AVB and TSN are more or less same technology, where TSN offers more deterministic services through ethernet. Currently, the AVB task group is renamed as TSN and the OpenAVnu stack supports TSN.
The summary of TSN components are given below. Most of the components are same as AVB, providing more reliability, low latency, better resource management and better synchronization.
AVB stack customization:
At NeST Digital, we have customized OpenAVnu AVB stack in different phases for developing talker and listener on various embedded platforms like i.mx6Q and i.mx8M for automotive applications, in infotainment area. The customized stack is verified with OpenAvnu stack (original version), NXP AVB stack, Renesas AVB camera module and AVB switches from NXP, Marvell and Renesas. The pre-certification testing is done with Spirent tool.
The media types used for custom AVB stack are: MP3 audio, H264 video, Raw audio from AM/FM tuner, Miracast stream from Android phone.
Issues faced:
Some of the issues that we faced during the OpenAvnu AVB stack customization are:
- MRP daemon was sending all frames, regardless of talker / listener and packet received or not.
- gPTP daemon was hardcoded for many parameters. Hence not able to execute for features like automotive profile.
- NXP stack expected the listener simulated a fast connect, even though it was not connected to the NXP talker earlier.
- OpenAvnu Listener was connecting only to one stream — audio or video at a time.
- gPTP daemon was not connecting to other gPTP domains to execute BMCA algoritm
- In i.mx8M listener, ~1 sec jump observed while playing video
- Stack had to stop and start every time a stream is completed.
- H264 camera data was unable to process by the OpenAVnu listener
- After reboot, delay was observed from i.mx8M talker to transmit data.
- i.mx8M talker was always sending the data through one queue, the FQTSS was not getting effective and standard data packets were suffering packet loss
Summary:
The blog is expected to give an overview of Audio Video Bridging, the key technologies behind the feature and the details of how an open source AVB code, sponsored by AVnu alliance was used to develop a custom AVB software stack for multiple hardware platforms with added feature sets.
