Sticky Note Detection and Extraction from Whiteboards
Whiteboards are a powerful tool for immediately capturing and arranging thoughts. Whether they are filled with mind maps, drawings or lumped together ideas — People interact with them during meetings, brainstorming sessions and especially throughout the whole Design Thinking process. But at some point, those sessions are finished and the results need to be documented for later applications.
It is not necessary to document the acquired information by hand. Instead, neXboard is the way to go. As pointed out in one of our former posts, by taking a picture of a group of sticky notes or even the whole whiteboard, the #Offline2Online feature creates digital counterparts for those elements by itself.
In order to recognize those elements, we use techniques from the field of Computer Vision by exploiting some common properties. Sticky notes are mostly colorful, have four corners and are of similar sizes and aspect ratios. On the contrary, the background is rather colorless and thus, we’re able to detect sticky notes by applying edge detection and contour matching. In the end, there is a separate image for each sticky note. We will look at further processing these images in the next step for better capturing the content in an editable format.
In the next step, images are digitized more detailed by splitting sticky note color and drawings. Based on the assumption that the largest region will be covered with the background color, the sticky note color can be extracted. Every spot with a color different from the main one can now be recognized as part of the drawing.
In order to create the drawings on the digital sticky note, they need to be vectorized. Therefore, strokes are skeletonized and their contour is used for determining the final vectors. Instead of transforming them into text, these vectors are kept as they are. As you can see in the above example, this works stunningly weel without any customization. Only for the sticky note containing the text “Design Thinking” the dots of the letter “i” are not recognized. This is because we need to be cautious about noisy pictures containing dirt or shadows.
By using lines instead of letters, it is possible to capture custom drawings and emojis. After importing the drawings to your Whiteboard, you can change their color and adapt or rearrange them.
The presented #Offline2Online feature enables people to easily document thoughts and further edit them even though they were originally on an analog Whiteboard. Later brainstorming sessions may refer back to these collected ideas and build on top of them. So what are you waiting for? Digitize your work!
If you are curious about whether neXboard could help you with your product development give it a try here. Test accounts are free for 30 days!
