DeepFaceDrawing — Neural Network That Turns Sketches Into People Faces
DeepFaceDrawing is a neural network model that generates an image of a person based on a sketch. The network was developed by researchers from the University of Hong Kong.
Live Demonstration
The problem of existing approaches
Existing neural network approaches to generating images from a sketch quickly synthesize facial images. However, such models are often retrained on sketches and require professional sketches or boundary maps to enter. To get around this limitation, the researchers propose to simulate a shape space with multiple face images and synthesize the image in this space to approximate the input sketch. The neural network uses a local-to-global approach. The model uses sketches as constraints. This allows for the generation of believable facial images.
What’s inside the model
The proposed approach consists of three submodules:
- A module where embeddings of key facial features are learned using separate autoencoders;
- Feature mapping network that decodes facial vectors into corresponding multi-channel feature maps;
- Image generation network
Model performance evaluation
The researchers compared the proposed approach with alternative existing architectures. Alternative models include Pix2pix, Lines2FacePhoto, Pix2pixHD, iSketchNFill. Below, in some examples, you can see that the proposed approach generates more photorealistic images.
