Tokyo Video Tech #3 Report (en)

Takesato Hayashi
Tokyo Video Tech
Published in
7 min readJun 5, 2019

Reported by Takesato Hayashi
Photograph by Akihisa Katsumi, Kanako Tomita

Our 3rd meetup event Tokyo Video Tech #3 Foundation (covering various fundamental technologies and applications for online video world.) sponsored by Fastly K.K. was held on 22 April 2019. This time we had 28 attendees to be gathered. There were three speakers, and one special guest from San Francisco. They provided a lot of slides, what they learned from events, activities, projects and insights. After the sessions, everyone enjoying “Networking with Food & Beverage” time, it was very special moments for online video engineers.

Event page:https://www.meetup.com/Tokyo-Video-Tech/events/260331664/

# Opening remark

Organizer Takesato Hayashi(Takesato) taking Opening remark part.

It was back to last December, when Takesato announced Tokyo Video Tech #1, he got an offer from Doug Chuchro from Fastly K.K. for supporting future event. Doug loves the style of engineering community which they are actively sharing experiences and what they learned. So, we start working together to create this event. This is the reason we’ve got very cool venue, Food & Beverage and special guest from San Francisco tonight.

And finally, Takesato announced upcoming Tokyo Video Tech #4 will be held at LODGE @ Yahoo! JAPAN 17F on Wednesday, 19th June, 2019. Details will be posted at Meetup.com and the Slack workspace <https://bit.ly/video-dev-jp>, keep in touch!

# Taipei Video Tech #3/Streaming Summit/NAB Show 2019

First speaker Hiromine Kanazawa(Hiromine), Forecast Communications & co-organizer of Tokyo Video Tech.

The session titled “Taipei Video Tech #3/Streaming Summit/NAB Show 2019”, he explained the event held in Taipei in March and held in Las Vegas this April.

Hiromine was there at Taipei on 13 March, as an expert to talk about Japanese Subtitles at Taipei Video Tech Meetup #3. The session titled “Subtitles University”. He explained Japanese specific rules like “Rubies” and “Tate-chu-yoko” first. Then he revealed his approach, combination of both WebVTT + Subtitle Image could provide looks good subtitles and clean video(can support multiple subtitle languages with just one video). I am hoping to have Japanese Subtitles related topics for Tokyo Video Tech sometime soon.

So, next topics. He traveled to Las Vegas to attend Streaming Summit at NAB Show 2019. I think many of you already knows what NBA Show is, so I will covering his Streaming Summit report.

Streaming Summit was 2 days event held at Las Vegas Convention Center.

URL: https://www.nabstreamingsummit.com/
Tickets: US$695.00
Sessions: 32
Speakers: over 75

He attend 4 good sessions there,
- Best Practices for Deploying CMAF, DASH and HLS at Scale
- Best Practices for Video Packaging, Playback and Low-Latency Delivery
- Engineering A Modern Super Bowl Streaming Workflow From The Ground Up
- Using VMAF, Netflix’s Video Encoding Metric to Measure QoE

*Session video recordings is now available. https://www.nabstreamingsummit.com/schedule/vegas19/

If you want to attend Streaming Summit, he encouraged to reserve 2 days only for this event. OTT related booth at NAB Show is located on South Upper floor, it takes 20 minutes from the summit to the booth. It is ultra hard to cover both areas. Plus, if you want to cover various topics, Streaming Summit provide some multiple tracks, so it is better to visit by two or more.

# Building a Network Emulation Application with Raspberry Pi

Next speaker Katsuyuki Sakai (Katz), SPARROWS & co-organizer of Tokyo Video Tech.

The session titled “Building a Network Emulation Application with Raspberry Pi”. While developing an OTT application, he encountered a big problem with corporate network. So, corporate network is too much high quality to test OTT application. He wants a device to emulate end user behavior to see what’s happening on the client side. This is the beginning of idea to creating a network emulator by himself.

Then Katz aware that he has no idea how does TCP packet loss and delay could affect effective throughput. So, here’s question.

How much packet loss do you think to make the effective throughput half?

1. 1%
2. 5%
3. 10%

The answer is 1. Only 1% of the packet loss makes your throughput in half! If CUBIC congestion control algorithm is used.

He creates a network bridge in Raspberry Pi. It controls network traffic between corporate network and device. You could manage Throttling/inserting additional latency/packet loss/data corruption/packet duplication. Comparing to high end network emulator, this system require only Raspberry Pi + USB-Ethernet. It is like buying a Toyota Prius or DIY(less than 200US$.)

Katz uses Linux “tc” command to control traffic. For your reference, if you want to follow his steps, please refer his slides below.

Also, he open sourced “Mr.Carson”, an easy to use administrative interface for Linux “tc” command with MIT License. Big thanks for your contribution!!

Mr,Carson.
https://github.com/katz/mr-carson

# Best Practices of building a highly scalable, available, resilient live streaming architecture

Last speaker of the day was Ashok Lalwani (Ashok), Lead for Solutions & Partnership, Video Streaming at Fastly.

The talk was titled “Best Practices of building a highly scalable, available, resilient live streaming architecture”. He discussed what he learned from across various large live streaming events.

Tonight Ashok’s topics are divided in three parts,

  1. Making your Live Streaming architecture resilient
  2. Invest in end to end monitoring
  3. Best practices to improve user experience

1. Making your Live Streaming architecture resilient

Ashok start showing Typical Live Stream Workflow. Origin + Single CDN + Video Playback QoS(Conviva/MUX/NPAW). Then subscriber growth happens, workflow will change. Typical first step is “Add more CDNs” with CDN Selector. You could improve Quality of Experience (QoE) and operation cost + complexity goes high.

Then again, to think about resiliency/availability of total workflow, Origin server and Encoder require redundancy. Ashok shows real architecture of couples of actual live events. Every component in the architecture is two. This is the next tradeoff, Total Cost of Ownership (TCO) vs Availability / Resiliency.

It is complex and hard to decide what is appropriate for your system. Ashok gave hint to this. Many of OTT services doing following things.

  • Maintain different architecture for Live linear/24x7 channels and Live events.
  • Manage their own live streaming architecture thereby allowing them to experiment and iterate.

2. Invest in end to end monitoring

The Internet is terrible, from your origin server to end user there’s too many intermediaries. And it is also wonderful, you could place some changes and immediately applied for mitigating problems of the service.

Ashok shows typical QoS monitoring metrics Rebuffer Ratio/Startup Time/Average Throughput and so on. But it’s not enough for live event. When some metrics goes high, it is hard to understand the reason why. He suggest to “Build your end to end data pipeline”. From end user, CDN Selector, CDN and Origin server. All of these information are important to see what’s happening on live events. Then you can start debugging and shooting issues. He shows monitoring example.

  • Client Side QoE
    - Conviva
    - MUX
    - NPAW
  • Synthetic Monitoring
    - TouchStream
    - Internal Alignment Checking Tools (based on hlspider)
  • Log Collection
    - SumoLogic
  • Dashboards & Slack Alerts

3. Best practices to improve user experience

Finally, he talked about commonly used techniques.

1. Bitrate Ladder and Video manifest variants for devices
Device-aware manifests is suitable for user experience and costs.

2. HLS or DASH or both???
HLS predominantly used in Live Streaming. Support DASH only if “DRM is a requirement” or “Device Support”.

Typically 4s or 6s segment duration used.

Finally,

The Networking with Food & Beverage time. All speakers and attendees were really enjoyed delicious food and drinks together. Asking a lot of questions to three speakers. Talking about today’s topics and media technologies.

I would like to take this opportunity to thank the speakers first. Especially Ashok, thank you for coming to Tokyo!! Awesome team member of Fastly K.K., discussions from the beginning, setting of the venue and delicious of Food & Beverage.

Thank you very much.

Slides

--

--