Improved Japanese subtitle support

ExoPlayer 2.12 adds parsing and rendering support for features often used in Japanese subtitles. We’ve received multiple requests from app developers to improve our support in this area.

We now support the following features from TTML and WebVTT sources:

• Vertical text (with tate-chū-yoko)
• Rubies

Netflix has written a blog post with more details on what these features are, and why they’re important in the context of Japanese subtitles.

Rendering support for these features in ExoPlayer’s SubtitleView (in the UI module) is opt-in. It can be enabled by passing VIEW_TYPE_WEB to SubtitleView#setViewType(). If you’re using StyledPlayerView or PlayerView, you can access the SubtitleView by calling #getSubtitleView().

Source representation

WebVTT uses the vertical cue setting and HTML-style <ruby> & <rt> tags. The ruby position can be controlled using the ruby-position CSS property and tate-chū-yoko is enabled with the text-combine-upright CSS property.

A WebVTT example with a ruby-annotated phrase displayed vertically (with tate-chū-yoko) then horizontally

TTML uses the tts:writingMode region attribute to control vertical text, and tts:ruby and tts:rubyPosition span attributes. Tate-chū-yoko is enabled with the tts:textCombine span attribute.

The TTML version of the example from above

Cue representation

ExoPlayer parses subtitle source files into Cue objects, which are then passed to TextOutput instances registered through SimpleExoPlayer#addTextOutput().

Vertical cues have Cue#verticalType set to either VERTICAL_TYPE_RL or VERTICAL_TYPE_LR (for right-to-left and left-to-right text respectively).

Rubies and tate-chū-yoko are encoded as custom style spans in Cue#text using RubySpan and HorizontalTextInVerticalContextSpan. These span implementations are ‘empty’, meaning they contain no styling or layout logic, because the native Android text components don’t support these features.

Rendering

Android’s native text components don’t support vertical text or rubies. We use a WebView instead, which allows us to leverage Chromium’s existing rendering support but is potentially resource-intensive. Requiring developers to opt-in with SubtitleView#setViewType() allows each application to weigh up the resource cost against the rendering benefits.

The view type can be toggled during playback, for example the WebView option can be enabled only when Japanese subtitles are selected by listening to Player.EventListener#onTracksChanged().

The first (vertical) cue from the examples above rendered using a WebView

The second (horizontal) cue from the examples above rendered using a WebView

With VIEW_TYPE_CANVAS (the default), ExoPlayer will do ‘best-effort’ rendering of these features as follows:

• Remove ruby text (before 2.12 the ruby text would be interspersed with the main text, which is completely unreadable).
• Transform vertical cues to a ‘logical’ position in a horizontal layout. This depends on the vertical writing direction (right-to-left or left-to-right). For right-to-left, vertical cues that would be on the right-hand side of the screen are rendered horizontally at the bottom of the screen (and left-hand cues are rendered at the top). Equivalently, for left-to-right text left-hand cues are rendered at the bottom of the screen and right-hand cues are transformed to the bottom.
• Centre the cue horizontally (Cue#position is ignored).

The vertical cue from above rendered without support for rubies or vertical text

Writing a custom renderer

ExoPlayer’s subtitle parsing logic is separate from rendering (with the Cue object used as an intermediate representation), so apps can use the included parsing logic and supply their own rendering logic. This may be useful for apps that want to display these features while avoiding using a WebView — they can write a custom TextOutput that interprets the Cue representation described above and draws the appropriate output to the screen. We would accept a high quality implementation following this approach back into the ExoPlayer project.

As always, please get in touch via our issue tracker if you have any questions or encounter problems. Thanks for reading!