Building VCode links with OpenCV and Branch.io
Hackaton Idea
In August 2017, Ellation held its very first Internal Hackathon.
We wanted to implement generation & reading of custom two dimensional barcode (VCode) image that can be read only by VRV application. The information inside the VCode would open a deeplink (start episode playback/open series screen and so on). The VCode would be able to be printed on any surface for reading by the VRV app.
By opening a specific screen (which will open camera) within the mobile app and pointing the camera to the image, user will be able to read the information encoded in the image and to start playing episode.
Image generation is implemented in the VRV web application as a separate module that can be used by marketing department to provide VCode with other branded commercials or anyone else who just wants to generate a shareable offline link of his favorite show on VRV.
Use cases
- A commercial could point you to a specific episode (it can start playing a new episode of your favorite anime series right away!)
- Offline marketing. The VCode could be used for anime nights and anime conventions. For example a sticker for your laptop, backpack or even print it on a t-shirt. The possibilities are are limited only by your imagination!
- Improve sharing. Too many steps — generate, copy/send, open. Now — only generate and scan!
How it works
VRV application uses Branch.io for deep-linking. Branch Link format: https://vrv.app.link/{id}. Id consists of 10 ASCII symbols. Each Ascii symbol — 1 byte (8 bits).
Encoding
The deeplink ID can have only case sensitive alphabetic and numeric characters which is total of 62. That could be encoded in a word of 6 bits length, and since we have 10 symbols it would be 60 bits of information. But for simplicity we went with ASCII. And since each ASCII symbol is 1 byte, we had to encode 80 bits of information. For example:
PNAdc9MlmD = 01010000 01001110 01000001 01100100 01100011 00111001 01001101 01101100 01101101 01000100
Each dot (rectangle/whatever) represents a single bit of information — presence of the dot in the position = 1, lack of the dot = 0. We will read dots from top to the bottom and then from the right to left. The matrix will be with constant size = 8(rows)*10(columns)
To encode a url we
- Generate branch link
- Encode id into matrix
- Generate image based on matrix, with each dots representing a cell from the matrix
Decoding
Decoding is the just the opposite of encoding. The encoding is linear, so we just need to make the same steps in the reverse order:
- Read matrix from the image by detecting dots
- Convert matrix into id
- Form a branch deeplink
- Open the deeplink
Image generation (id -> VCode)
Image generation was implemented as a simple web-page.
We implemented the algorithm that consumes the matrix described in the Encoding paragraph and generates the image based on the matrix. Generated images are expected to differ only on the presence of dots, based on the matrix.
Image decoding (VCode -> id)
Image positioning
First of all we need to find a VCode on the picture. VCode location detection was implemented using OpenCV and Rectify library. Rectify was used to find the biggest yellow square on the picture and then transform the image from 3D projection to 2D square.
Image decoding
Image decoding was done by detecting if a certain dot is present. We achieved this by converting the VCode rectangle to monochrome and then splitting it into small rectangles each representing a single bit in the matrix.
Design
The design had to be done first. We had to agree on the shape of the image, colors we would use and the way we encode information. We wanted the colors to have high contrast, so we could distinguish the shapes in monochrome on decoding step. Thankfully main VRV colors were good enough for us.
Next step was to divide yellow space in small squares. Each square could contain a dot, which is a small version of VRV logo:
A square with a dot is considered to be “one”, “zero” — otherwise. Now we have a grid with 10 squares on each axis:
Considering logo, we had 84 available squares for encoding which is more than enough.
Each byte we stacked vertically one by one.
Summary
Even though our team didn’t win the hackathon, we had a great time coding and building the project. Check the sample video bellow of the working prototype in action.
VCode Generation: https://codepen.io/iparm/full/PKELWb
