Coding Period Week 3(21st June- 28th June)

Meeting Minutes-:

• In the meeting we discussed about the results of the RHA on the lab interview dataset.
• Prof. Mark suggested to define some basic specs which a video must have for RHA to work efficiently.
• Goals for this week — Main goal of this week is to detect and remove flickers in the anonymized video.
• During the meeting, I proposed a suggestion to modify faces in frames that were not properly anonymized. My idea was to use generative inpainting to remove a portion of the face and redesign it. However, Daniel raised an important point. He mentioned that generative inpainting may not preserve the facial expressions, making it comparable to using a face hider.
• Final algorithm -: We came to a decision to remove faces from frames where the swapper was not performing effectively. However, for the frames where the swapper was working well, it was agreed to leave them unchanged.

Progress-:

• This week started with going through the RHA code and understanding the codeflow.
• Just a heads up, if you make any changes to RHA, you won’t see them take effect unless you add the file’s path to the PYTHONPATH environment variable. To save you from going through the same debugging process I did, here’s a command you can use while you’re in the RHA main directory. It adds all the subdirectories to the PYTHONPATH variable. Trust me, it’ll save you a lot of time and frustration.

export PYTHONPATH=.:./subdir:$PYTHONPATH

• I encountered another challenge in creating a complete conda environment that could successfully run both the swapper and hider. To address the issue of module incompatibilities, we followed the approach we discussed in our previous blog post, where we created two separate conda environments.
  Here’s a helpful tip : Start by installing the dependencies needed for the swapper and then proceed with the hider. You can find detailed instructions in this link link. Make sure to install this version of pytorch
  The .yml file of the environment has been stored in the home directory under the name of complete_rha.yml .
• Algorithm-:
  1) To detect flickers we will traverse original video and video anonymized by fsgan frame by frame.
  2) We’ll detect the faces in both frames.
  3) Then, we’ll calculate the Mean Squared Error (MSE) between the corresponding faces.
  4) If the error is below a certain threshold value, it indicates that the face is not properly anonymized, and we need to hide it. On the other hand, if the error exceeds the threshold, we can store the face as is, without any modifications.
• This algorithm seems to produce promising results even on the videos on low resolution videos. If everyone agrees, I will add this to the final code.