How Handlebars Helps Power Content Metadata Syndication
Context
The Content Delivery Engineering (CDE) team is responsible for the delivery of video metadata (titles, descriptions, images, cast/crew, rights/restrictions, playback URLs, etc.) to external destinations outside of Disney Streaming Services (3rd party catalogs, ad exchanges, partner CMS’s, etc.).When our team was tasked with building a new application in support of that charter, we aimed to build it in a way that would offer the least amount of maintenance and overhead as possible. This includes being able to support most new syndication use cases without the need for any code changes. Additionally, we wanted to avoid writing our own templating language but rather use an existing, proven framework to power our templates. Lastly, each partner had different requirements around the data they wanted delivered which required lots of custom logic. We wanted to ensure this custom logic for each partner did not exist in our code base but rather was contained in each partner’s particular template. Enter Handlebars!
Handlebars to the Rescue!
Handlebars.js is a powerful templating engine that allows you to write templates to transform data into arbitrary formats. It is built on the Mustache Templating Language but offers additional functionality. Features such as evaluation of conditional statements and array iteration are extremely helpful when building out templates.
One of the limitations is that Handlebars is a javascript framework and comes packaged as a npm module. Our application was designed to be a Scala application with multiple components. The requirements laid out for our application were:
- Deliver content metadata to external systems in various formats required by those systems. A concrete example of this is the need to syndicate video metadata to an ad server to enable relevant ad serving in our ad-supported products.
- Be able to support json, xml and plain text outputs. Currently we have approximately 20 different feeds we deliver to external partners, all with different formats.
- Push the final output to a service endpoint, S3 bucket or Kinesis stream. In addition to push-based protocols, we needed to be able to stand up HTTP endpoints from which consumers could pull data.
Solution
After we decided that Handlebars was the framework of choice, we began looking for a way to integrate it into our Scala project. We discovered this Scala library which allowed us to leverage most of the basic helpers which makes Handlebars so powerful. It currently implements version 1.0.0 of the original Javascript version. The library also lists what is not supported compared to what is supported by the original Javascript implementation here. Below is an excerpt of one of our templates using a few of the basic handlebars helpers:
{{#if data.photos}}
<bam:Images>
{{#each data.photos}}
<bam:Image xlink:href="{{uri}}" height="{{height}}"
width="{{width}}" type="image/jpeg" key="{{imageKey}}"/>
{{/each}}
</bam:Images>
{{/if}}The above template is checking if the photos array exists. If it does, it will create a <bam:Images> object. Then, using the #each helper, it will iterate through the photos array and add the values in the Photo object to the <bam:Image> attributes. If we tried writing code to handle this logic in our application directly, we would end up with lots of bespoke logic based on the partner’s desired output. Using templates with helpers as illustrated above allows us confine all this logic to a particular template for each partner.
Here is the Scala code for the #if helper in the Scala library:
def `if`(obj: Object, options: Options): CharSequence =
obj match {
case it: Iterable[_] =>
if (it.isEmpty) options.inverse() else options.fn()
case _ =>
IfHelper.INSTANCE(obj, options)
}The photos field is an array so it’ll match on the initial case, then it does a simple check if the array is empty or not. If it is empty, it’ll return options.inverse() which actually tells the template to use the value that the template has defined in the “else” block. Else it’ll call options.fn() which tells the template that the condition evaluates to true and to use the value in the “if” block.
Finally, the output of the above block will be:
<bam:Images>
<bam:Image key="" type="image/jpeg" width="133" height="200"
xlink:href="https://images.unsplash.com/photo-1530143584546- 02191bc84eb5"/>
</bam:Images>We Need More!
In our effort to remain flexible and stick to the basic tenet that as much custom logic as possible should live in the templates instead of the application code base, we had to begin extending the referenced library with our own helpers.
A good example of this is the custom #in helper we created. The logic it applies is if any of the values specified the comma delimited string exist in the field specified, then it should evaluate to true. Here is an example of the #in helper in use:
{{#in system "FCC-TVPG (USA), TVPG"}}
<advisory systemCode="us-tv"/>
{{/in}}In the above, it is simply checking if the field system contains any one of the values specified. If it does, it should continue into the block below and add the advisory element into the output.
Here is the Scala code we wrote to support this #in helper:
def in(sourceObject: Object, field: Object, values: Object, options: Options): CharSequence =
(sourceObject, field, values) match {
case (str: String, list: String, _) =>
if (list.split(", *") contains str) options.fn() else options.inverse()
case (sourceObject, field: String, _) =>
in(Try(sourceObject.toString).getOrElse(""), list, null, options)
case _ =>
options.inverse()
}The logic is pretty straightforward and writing this single helper has helped us immensely as it is used in the majority of our templates. Of course as we continue to create more templates, we add additional helpers as needed.
A slightly more complex helper would be our #dateCompare helper which allows us to check if a date field is before, equal or after a certain date we specify. Below is an example of it in use:
{{#dateCompare startDate "isBefore" "2019-01-31T00:00:00Z"}}
<matchTime="{{startDate}}"/>
{{/dateCompare}}The field we’re comparing on the source object is the startDate. If the start date is before 2019–01–31T00:00:00Z then we can go ahead and set the matchTime element. As you notice, we have an additional argument in our expression compared to the #in helper. This allows us to specify whether we want to use isBefore, isEqual or isAfter. The key to this is in our Scala helper function, we just needed to add additional arguments in our function signature. In fact, you can specify any amount of arguments you want in the any Scala template helper you write. The code for this particular helper is below:
def dateCompare(dateObject: Object, operatorObject: Object, compObject: Object, dateFormatObject: Object, options: Options): CharSequence = {
val (dateValueOption, compValueOption): (Option[LocalDateTime], Option[LocalDateTime]) = (dateObject, compObject, dateFormatObject) match {
case (dateField: String, compDateField: String, format: String) =>
(DataExtractor.getDate(dateField, format), DataExtractor.getDate(compDateField, format))
case (dateField: String, compDateField: String, _) =>
(Try(Some(LocalDateTime.parse(dateField, DateTimeFormatter.ISO_DATE_TIME))).getOrElse(None), Try(Some(LocalDateTime.parse(compDateField, DateTimeFormatter.ISO_DATE_TIME))).getOrElse(None))
case _ =>
(None, None)
}
val result: Boolean = (dateValueOption, operatorObject, compValueOption) match {
case (Some(dateValue), "isAfter", Some(compValue)) =>
dateValue.isAfter(compValue)
case (Some(dateValue), "isBefore", Some(compValue)) =>
dateValue.isBefore(compValue)
case (Some(dateValue), ("isEqual" | "==" | "equals"), Some(compValue)) =>
dateValue.isEqual(compValue)
case _ =>
false
}
if (result) {
options.fn()
} else {
options.inverse()
}
}The code block above allows us to compare whether a date in the source object isBefore, isAfter or isEqual a date that we specify. It also allows us to specify a date time format. If nothing is passed, it defaults to use standard ISO Date time format.
Below are just a few more of the custom helpers we have implemented:
#inas referenced above#notInwhich is simply the inverse of the#inhelper#startsWithwhich is used to match on any string starting with a specified prefix#xmlDurationwhich is used to convert a duration in milliseconds to xsd:duration format- and several more!
From the list of helpers above we can see that there is wide range of functionality that can be added as a handlebar helper. This allows us to use the same helpers across handlebars templates regardless of the output format we’re building.
Putting it all Together
Now that we have created all the helpers we need and created templates that match the requirements outlined by our partners, how do we put it all together to actually produce the expected output? The Scala library we’re using is written on top of this Java implementation. To get started, we simply create a new instance of the Handlebars object.
val handlebars = new Handlebars()Then we can go ahead and start registering your helpers. The Scala library offers a few Scala Helpers, and of course we want to add the ones we created. We can do that by chaining the registerHelpers function to the above:
val handlebars = new Handlebars()
.registerHelpers(ScalaHelpers)
.registerHelpers(inject[CustomHelpers])Now that we have our handlebars instance, we can begin building outputs. First thing we do is actually pass in the handlebars template we created:
val template = <some handlebars template>
val handlebarsTemplate = handlebars.compileInline(template)where template equals:
{{#if data.photos}}
<bam:Images>
{{#each data.photos}}
<bam:Image xlink:href="{{uri}}" height="{{height}}"
width="{{width}}" type="image/jpeg" key="{{imageKey}}"/>
{{/each}}
</bam:Images>
{{/if}}Now that we have our actual template, we can pass in the source data which in our case would be json, and voila, we’ll have our output.
def ctx(obj: Object) = Context.newBuilder(obj).resolver(Json4sResolver, MapValueResolver.INSTANCE).build
val output = handlebarsTemplate(ctx(contentJson))where contentJson equals:
{
"photos":[
{
"uri":"https://someurl1.com/",
"height":608,
"width":1920
},
{
"uri":"https://someurl2.com/",
"height":1024,
"width":780
}
]
}The ctx function defined above creates the context stack for the template, this makes all elements in the source objects available for the template. We then use this function and pass it into ourTemplate object. This then merges the compiled template with the source context and voila, we have a fully built output!
The following is the final output that will return from our call to merge the compiled template to the source context.
<bam:Images>
<bam:Image xlink:href="https://someurl1.com/"
height="608" width="1920" type="image/jpeg" key=""/>
<bam:Image xlink:href="https://someurl2.com/"
height="1024" width="780" type="image/jpeg" key=""/> </bam:Images>Where are we now?
As our application has continued to mature, we are now at a place where adding a new integration is as simple as creating a new Handlebars template and mapping the source data to the desired output. This has allowed us to introduce new use cases at a rapid pace without the need to write custom syndication applications for each. Our application plays a crucial role in the Disney Streaming Platform and the use and customization of Handlebars has made it relatively easy to manage our growing consumer base.
