Image-to-image constellation outlining

Figure 1. Connecting the dots [1]
Figure 2. Example of a llama constellation and its respective outline
Figure 3. Constellation of lama at different difficulties
Figure 4. GAN architecture
Figure 5. Objective function of the pix2pix model
Figure 6. Inverse mapping function of the CycleGAN model [4]
Figure 7. Effect of epoch on pix2pix model behavior.
Figure 8. First row shows the the real image of an accordion and constellations with different difficulties, the following rows show the respective model outputs (there were no images for noise level 002 at difficulty 4)
Figure 9. Plane output from CycleGAN (noise level 003) at difficulty level 4 (the easiest) — initial constellation, real image and fake image shown in the gif
Figure 10. Plane output from CycleGAN (noise level 003) at difficulty level 17 (the most difficult) — initial constellation, real image and fake image shown in the gif
Figure 11. Distribution of objects per category in the training dataset (right) and evaluation dataset (left)
Figure 12. Classifier accuracies of evaluation dataset images from different models (reminder: NaN value due to having no constellations with difficulty level 4 and noise level 002)
  • to improve the accuracy of the validation classifier — use contrastive learning in the transformer part and train it with human sketches (general properties and feature learning), train the classification part using the same network with outlines
  • test out →NEW← improved version of cycleGAN model — Contrastive Unpaired Translation by the same authors (Taesung Park and Jun-Yan Zhu) — supposedly faster and less memory-intensive [6]

References

[1] https://www.upload.ee/image/7553119/connect-the-dots-fish-300px.png

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store