Let’s talk about video bookmarking
Lots of customers upload videos to Box and Box offers a great preview experience for most video formats. Often customers want to direct the viewer to a certain time in the video and this is mostly done via email/Slack/Teams with with a given time for the viewer to jump to. Like this…
Check out the recording from our Q2 company all hands
- Intro from John: 0.12
- Update from Finance Team: 3.32
- Product news with Tim: 6.45
- Quiz and fun time: 10.12
This article shows a way of saving the times directly against the video in Box so all users can jump directly to the important parts of the video without having to find the email or Slack message with the timings. AI can obviously play a role here, but in this article I will show you how, with a simple integration, you can allow a user to manually add bookmarks to videos. So instead of having to find emails/Slack messages with a list of times users can see the bookmarks directly in Box and jump to the correct time in the video. Once the bookmarks have been set, any user previewing the video will have access to them and they can be updated directly on the file.
Here is an example of what it would look like for the end user when they access a video with bookmarks.
So how does it work?
A Box Skills Card is a multimedia type display of metadata that can interact with the video previewer, meaning you can add time stamps to text and clicking on these will move the video replay to the given time. This is available for all users in Box but will only appear in the UI when the file in question has a SkillsCard saved in metadata.
What I will show in this article a single page OAuth 2.0 integration app that will allow user to interact with the video to set bookmarks and save these back to the Skillscard using Box API. It consists of an authentication backend and a jquery/html front end.
Backend
In this example the backend is deployed as a NodeJs based AWS Lambda function with a public API gateway, but the functionality is certainly not limited to AWS Lambda and you can use the application architecture that works for you. Here is an example of using a Node server and here a Python example achieving the same thing.
The backend consists of a standard 3-legged OAuth code to token exchange. This will take the OAuth2.0 auth code and exchange this into a token and a refresh token. The reason this is a backend process is that the token exchange requires a client secret which shouldn’t be publicly available.
Here is a sample of the OAuth2.0 code flow in an AWS Lambda function with a handler function
const BoxSDK = require("box-node-sdk");
exports.handler = async function (event) {
//Auth code from querystring
const authCode = event.queryStringParameters.authCode;
//Client ID from querystring
const clientId = event.queryStringParameters.clientId;
//Client secret that matches the client ID
//Read from environment variable as an example
let secret =process.env.BOX_APP_SECRET;
var tokens;
var sdk = new BoxSDK({
clientID: clientId,
clientSecret: secret,
});
//Get the tokens from Box using the Auth code
await sdk.getTokensAuthorizationCodeGrant(
authCode, null, function (err, tokenInfo) {
tokens = {
token: tokenInfo.accessToken,
refreshToken: tokenInfo.refreshToken
};
}
);
//Return the tokens to the caller
const response = {
statusCode: 200,
headers: {
"Access-Control-Allow-Origin": "*", // Required for CORS support to work
},
body: JSON.stringify(tokens),
};
return response;
};The token is then available for the front end, scoped to only be able to interact with the file the integration was invoked for.
This is what happens when the page is loaded. The page is loaded as a browser redirect once the user has authorized the app in Box. The URL will have an auth code in the query string parameters that can be exchanged to a token and this is what is passed to the backend function.
$.ajax({
method: 'get',
//Use URL to above lambda function
url:"https://<URL TO TOKEN EXCHANGE FUNCTION>",
//Data passed is coming from box as query string parameters
//authCode, file ID and clientId
data: {
authCode: params.get('auth'),
id: params.get('id'),
clientId: params.get('clientId') },
crossDomain: true,
cache: false,
//Show a loader gif while waiting for token
beforeSend: function () {
$('#loader').show();
$(".container-fluid").hide();
},
//Show the main page on completion
complete: function () {
$('#loader').hide();
$(".container-fluid").show();
},
success: function (response) {
//Get the token from the lambda
gaccessToken = response.token;
//Load the existing bookmarks
loadBookmarks();
$("#preview-container").show();
//Load video in the Box UI Element preview
preview.show(boxFileId, boxAccessToken, {
container: '.preview-container',
showDownload: false
});
preview.addListener('load', (data) => {
//Get the duration of the video from the player
//The duration must be sent as part of the skills card payload
duration = preview.viewer.mediaEl.duration;
$("#save-bookmark").prop('disabled', false);
});Front end
The app is reached via the the ‘Integrations’ context menu in the Box Webapp for video files. More on how to configure this part later
The invocation of the ‘Videobookmark’ integration will launch a screen like below.
The front end in my example is a ‘simple’ HTML/Javascript page that loads the video preview, a control to add bookmarks and a list of existing bookmarks that also allows removal. Clicking on Save will save the bookmarks back to the file using Box Metadata API.
When adding a bookmark the user needs to enter a time or simply start/stop the video at the place and the time will autopopulate. The end time will automatically be 10 seconds after the start but you can change this manually as well.
The bookmarks are saved in a transcript card which is part of the SkillsCard.
Getting the existing bookmarks is done by the below snippet using the Box Metadata API to fetch the values from the SkillsCard and populating them in a list on the HTML page. The SkillsCard is accessible to all Box customers as a global metadata template. The SkillsCard is stored as a JSON structure in a metadata field called ‘cards’.
function loadBookmarks() {
//Call the Box metadata API to fetch the existing metadata
$.ajax({
method: 'get',
url: "https://api.box.com/2.0/files/" + boxFileId + "/metadata/global/boxSkillsCards",
crossDomain: true,
async: false,
headers: {
"Authorization": "Bearer " + boxAccessToken},
cache: false,
success: function (response) {
//Set the hasMetadata flag as the create and update calls are different
hasMetadata=true;
//Add the bookmarks to the page
$.each(response.cards[0].entries, function (k, data) {
//Each bookmark is added to an HTML list with some bootstrap styling
$(".bookmarks").append(
"<li uid=" + i + " id=v_" + i +
" class='list-group-item d-flex justify-content-between align-items-center ital' " +
" bk='" + data.text + "'><span style='width:100px'>" + data.text + "</span>" +
"<span id='start_" + i + "' class='badge badge-primary badge-pill'>"
+ fancyTimeFormat(data.appears[0].start) + "</span>" +
"<span id='end_" + i + "' class='badge badge-warning badge-pill'>"
+ fancyTimeFormat(data.appears[0].end) + "</span><span class='badge badge-primary badge-pill remove'>x</span></li>")
i++;
});
//Add remove handler for the bookmark control
$(".remove").click(function (event) {
//Remove bookmark
$(this).parent('li').removeClass('d-flex').hide('slow');
//Show save button as a change was made
$("#saveAll").show();
});
}
});Resulting in this list on the page:
Once bookmarks have been added/removed they can be saved back to the file using this code:
function saveBookmarks(data,duration,hasBookmarks) {
//Use put if bookmarks already exists, POST to create
var method = hasBookmarks?"PUT":"POST";
$.ajax({
"async": false,
"crossDomain": true,
"dataType": "json",
"url":"https://api.box.com/2.0/files/"+boxFileId + "/metadata/global/boxSkillsCards",
//The input for create and update metadata are different so a transform method is used to format curretly based on
//whether it is an update or a create
"data": transformData(data,duration,new Date().toISOString(),hasBookmarks),
"method": method,
"headers": {
"Authorization":"Bearer " + boxAccessToken,
//The content type is different for update and create
"Content-Type":hasBookmarks?"application/json-patch+json":"application/json"
},
success: function (response) {
//Reset the bookmark adding controls
//and enables regular control buttons after the save
resetControls();
}
});The ‘transformData’ method simply formats the data based on whether I am updating or creating the bookmarks. For the data I use a ‘skeleton skills card’ which has placeholders for the bookmarks data, the video duration and timestamp. The SkillsCard requires a certain format for the JSON structure so by using a skeleton card I am ensuring the format.
function transformData(data,duration,timeStamp,hasBookmarks) {
if(hasBookmarks) {
//For updates the format is json+patch. The 'cardBootStrapUpdate'
//contains a skeleton card with placeholders for data, duration and timestmp
var card = cardBootstrapUpdate.format(data,duration,timeStamp);
return '[{"op":"replace","path":"/cards/0","value":' + card+'}]';
}
else {
//For create we create the entire card as payload.
//The 'cardBootStrapCreate'contains a skeleton card with placeholders for data, duration and timestmp
return cardBootstrapCreate.format(JSON.stringify(data),duration,timeStamp);
}
}Please find the full HTML page and javascript GitHub repository with some instructions on how to implement this yourself.
https://github.com/pchristensenB/box-videobookmarking-demo
Configuring and enabling the integration
To get the integration working in your Box environment there are a couple of steps
- Create the OAuth 2.0 app and the web integration using this guide:
For application scopes for the OAuth 2.0 add read and write files and configure the Cors hosts with the web site host you are going to use
For the web integration full permissions are required as we are updating the file and it only needs to be scoped to access to the file/folder it was invoked on.
The Callback configuration should point to the web server where the single page app is deployed. This can be public or private but must be https. In this example the clientID of the OAuth 2.0 application is passed a parameter to the page and used in the OAuth flow to exchange the token.
Finally the visibility of the integration is set to ‘Online’. The integration will initially be available only for you as the developer.
You can share it with others if the target group is small. To share, go to ‘Apps->My Apps’ from your main Box view and click on the ‘Videobookmark’ app (this is the name you gave the integration you configured above). From the bottom of the page you can copy a link and share with other users to enable the app on their instance.
If you want it more widely distributed, eg. to your entire org or other orgs, you can add it to the app center.
Here is a link to a GitHub repo with all the code and instructions to set this up.
