Google Summer of Code: Week 7 - 8 Overall Highlights extraction
This was the most awaited part of the project. The part where all the pieces come together to make it happen!

So the Curves I had were:
(1) Cuts Density Curve [red]
(2) Motion Activity Curve [green]
(3) Sound Energy Curve [blue]

All these smooth curves have distinct peaks that marked some important event. For an event, since the peaks from the three curves were not aligned, the high excitement moments were to be merged to a final curve.
The first attempt was to try out with the mean of the curves. But first, the curves were weighted to be maximally selective and to search for highlights in the segments containing high local maxima of the time curves of all three excitement components. The weighting function used was:

Then averaging out the curve and finding the peaks, it was found that the this method was approximating the events with a lot of error. The detected curves were more than ~100 secs away from than the actual curves.
Wait, what? But why? All the three curves were meant to help get the final highlights accurately, right?

It turned out that I was smoothing out the curves too much, losing a lot of important data. Also that averaging the curves was not a pretty idea.
So, the next thing that I tried was to take logical AND of the curve, as a replacement to averaging. That was done by multiplying the values of the three curves for every frame, as multiplication can be considered as an AND operation.
Thus the original rough curves were multiplied and then the total was smoothened out with a kaiser function with smaller length parameter. This gave us a curve:

Selecting ~ top 90% of peaks, gave the 2 goals that took place. The peaks were marked at [2428, 2901, 6085, 10370, 28267]. (Note: fps=10)
- The Goals took place around frame_number = 6000 and 28100.


- Also at around frame number 10500 a close goal attempt was made which also led to a handball foul, and a free kick.

- The match started at around frame number 4000, so the 2428 and 2901 peaks are due to pre-game excitement.

The delay in detection can be seen of ~8–10 seconds, which is acceptable given the fact that the excitement increases after the event happens.
So it was a success right?

For now, YES. Although, it needs to be tested for its robustness on different qualities of videos, different sound volumes etc. The model may also need some more tweaking of parameters depending on how selective the user is while expecting highlights of the sports match.
But, for now… Yaayyy

What’s up next?
The next thing to be done will be to test this excitement model, and create out a proper package of it, cleaning and documenting the code properly. Also in the final month, I’ll try to build a good graphical interface for the software.
Thanks for reading.