week 3 | GSoC’19 | CCExtractor Development

Image Source: https://ccextractor.org

This week I started working on UI side, But soon I realized that the core part of the project needs lots of improvement and UI part can be done later on may be in between last few weeks or better after GSoC period. Therefore I particularly started working on this core part. This week I achieved a great milestone in especially in video processing part by introducing a simple local cache memory for the embeddings. You can check the problem I faced and the solution mentioned on GitHub kanban board . Here local cache means, I maintained a local dictionary to the function in which all the faces and embeddings are kept. First, it checks whether the embeddings of the faces in current frames were present in the previous frames or not. If there is a hit then it moves to identify the next face else It looks globally for the face embeddings and on hit, current face embedding is added to the local cache. This simple solution gave a huge boost in performance.

Let me show you result on this youtube video,

Video metadata:

fps : 29.97002997002997 
total_no_of_frames : 2188.0 
duration: 73.006267 seconds
frame_hops : 2 
min_frame_dur gap in ms : 300.3000013711151

output:

Time taken by previous method : 782.4210019111633 seconds

output without cache

Output

Time taken by new method : 656.1480021476746 seconds — 

This is video has two main characters, 1) Elon Musk, 2) Unknown.

You can clearly see the difference in output accuracy of detected faces. The current database had 1295 embeddings.

You can find the patch for this feature in #PR11

Concurrently I also worked on APIs part and also improved the code quality for the APIs service.

If you really enjoyed this post and also following my earlier blogs then I’m sure I’m pretty lucky. If this project intrudes you and want to contribute then join the CCExtractor channel and have a discussion :) and stay tuned 😎.

Profile links LinkedIn GitHub Twitter