Building FabBits : GSoC with CCExtractor #2

These last three weeks I worked on soccer goal detection and making the action sequence detector better.

My approach for soccer goal detection was basically this -

• Segment the scoreboard.
• Monitor the changes in scoreboard for detecting goals.

Turns out the scoreboard area has quite a bit of edge content owing to the textual regions it has. Due to this, Sobel/Scharr filter outputs show a better imprint of the scoreboard area and also weeds out unnecessary details. So, basically, what I had to do was pass a random frame, given it has the scoreboard, through Sobel filter and get the contours. Right? Actually, no.

This is what happened when I passed a random frame in -

“Ahhh, the spectators! If only they were not be in the frame!” — yours truly.

I got around this by searching a sub clip, ~10 min, of the video for frames with least edge content and used the best one for further processing. Below are some frames with low edge content —

Original frames and their Sobel filter output.

Then, after blurring and morphing, I found the contours in the least-edges, LE henceforth, frame, got their bounding boxes, and marked the one as the scoreboard that satisfied a certain criteria for area and aspect ratio.

And, there we have our scoreboard!

However, there is still an unnecessary thing that can pose problems later — the timer. Since my approach is based on comparing changes in scoreboards, the fact that there is something in there changing every second is pretty bad.
So, to remove the timer, I calculated the structural similarity between the LE frame’s scoreboard and the scoreboard of the frame that comes after a second to the LE frame. And since the only thing that changes in such a short while is the timer, I was able to remove the timer side from the scoreboard.

Whole scoreboard in left; score-only area in right

The next step is monitoring the score-only area for changes. I used structural similarity again for getting the differences between two scoreboards. This is how it looks when the score changes -

The rightmost image shows the structural similarity between the two scoreboards. The green areas signify low similarity.

Then it was just a matter of iterating through the match and comparing the score-only area for significant changes. Although, choosing a threshold for similarity, whose breach would mean a goal, is another problem by itself, based on four matches from different studios, I found out that 87.5% is a good bet.

And that’s how you get goals from soccer!
Check out and contribute to FabBits if you have time!