Converting an Anime Image to a Sketch
Edges are sudden changes in pixel intensities within an image. So, to detect the edges, we can go looking for pixels with sudden changes in the neighboring pixels.
I want to convert a sketch of an anime character so that my nephew can color it. As you will see, I am going to use Python to accomplish this conversion. Here is the anime image onto which we want to do our magic:
The anime series is called Naruto — I think it is one of the most popular Anime series in the world. But the photo is from Uchiha Madara — one my favorite characters.
The edge detection algorithm I will be using involves (1) conversion to grayscale (2) noise reduction, (3) gradient calculation, and (4) thresholding. Well, let’s do this:
(1) Conversion to Grayscale
First, import packages that we are going to use:
import os
import cv2
import matplotlib.pyplot as pltNext, let’s find the image on our computer memory and read the image into our program:
path = r’C:\User\Documents\gerdoo’
os.chdir(path)
file = path + ‘image.jpg’
image = cv2.imread(file, cv2.IMREAD_UNCHANGED)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)The last line in the above code converts the image into a grayscale version of the image. Note, however, that the cv2 function is BGR2GRAY and not RBG2GRAY. That is because cv2 by default stores the image channels in blue-green-red (BGR) order (not the more common red-blue-green (RBG) order). Next, let’s plot them side by side:
fig, ax = plt.subplots (1,2, figsize = (10,5))
ax[0].imshow(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))
ax[0].axis(“off”)
ax[1].imshow(cv2.cvtColor(gray, cv2.COLOR_BGR2RGB))
ax[1].axis(“off”)
plt.show()The result should look like this:
(2) Noise Reduction
This step is done to reduce noise in the image for better results. We can smooth the image by applying a Gaussian Kernel to the image. Gaussian kernel can have any size (you can think of it as a convolutional filter). Smaller kernels are less visible and larger kernels can be more effective. Let’s do it:
blur = cv2.GaussianBlur(gray,(3,3),cv2.BORDER_DEFAULT)Single line of code, isn’t that amazing. Although python packages like OpenCV and SciPy have made this easy for us to implement, this is really not that simple of a process. Read more from Refs 3–5.
We can’t see much of a difference in this image, because it is fairly simple image.
(3) Gradient Calculation
I think gradient calculation is the core of most edge detection methods. Gradient of an image can be defined as a directional change in image intensity. I come from a physics background and understand gradient as an operator that vectorizes a scalar distribution in direction of steepest/fastest increase. Mathematically, you can think of it this way:
Enough math already. Sobel and Scharr are two well-known gradient calculation methods and can be conveniently implemented using OpenCV. I am using Sobel here (use Scharr when you want more information content in the sketch).
grad_X = cv2.Sobel(gray, ddepth=cv2.CV_32F, dx=1, dy=0, ksize=-1)
grad_Y = cv2.Sobel(gray, ddepth=cv2.CV_32F, dx=0, dy=1, ksize=-1)Note that ‘grad_X’ compute the gradient across the x direction and ‘grad_Y’ compute the gradient across the y direction. But they are both float numbers and we should convert them to 8-bit integers (between 0 and 255). Like this:
grad_X = cv2.convertScaleAbs(grad_X)
grad_Y = cv2.convertScaleAbs(grad_Y)Here, we are interested in a combined X and Y gradient. We can combine them like this:
combined = cv2.addWeighted(grad_X, 0.5, grad_Y, 0.5, 0)And I did play around with the kernel sizes and decided that I like the blur kernel to be (3, 3). You can see some my experimentation in the figure below:
(4) Thresholding
We can already see ‘strong edges’ of the original image. Those are edges that essentially define boundaries of physical object and do not contain unnecessary texture information (unnecessary for edge detection algorithms). We can use thresholding to emphasize strong edges and drop weak edges. Like this:
ret,threshold_binary = cv2.threshold(gradient,100,255,cv2.THRESH_BINARY_INV)Thresholding is one word and I have done it (with power of open source Python and its amazing community) in one line, but it is a huge topic in itself. Here, I used binary thresholding and loving how the end-result looks:
Almsot forgot to mention; you can generate the above subplot using Matplotlib:
fig, ax = plt.subplots (2,2, figsize = (10,6))
ax[0, 0].imshow(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))
ax[0, 0].axis(“off”)
ax[0, 0].title.set_text(“image”)ax[0, 1].imshow(cv2.cvtColor(gray, cv2.COLOR_BGR2RGB))
ax[0, 1].axis(“off”)
ax[0, 1].title.set_text(“gray”)ax[1, 0].imshow(cv2.cvtColor(gradient, cv2.COLOR_BGR2RGB))
ax[1, 0].axis(“off”)
ax[1, 0].title.set_text(“gradient”)ax[1, 1].imshow(cv2.cvtColor(threshold_binary, cv2.COLOR_BGR2RGB))
ax[1, 1].axis(“off”)
ax[1, 1].title.set_text(“threshold_binary”)
This was not a difficult thresholding exercise. More complex images will demand more sophisticated algorithms to handle complexity of information content in those scenarios. I am hoping to write about them next.
Thanks for reading! If you liked it, please let me know by clapping, sharing, leaving a comment, following me, or sending me a message.
References:
[1] Sobel Derivatives, OpenCV, https://docs.opencv.org/2.4/doc/tutorials/imgproc/imgtrans/sobel_derivatives/sobel_derivatives.html
[2] Schar, rOptimale Operatoren in der Digitalen Bildverarbeitung, http://archiv.ub.uni-heidelberg.de/volltextserver/962/
[3] Sofiane Sahir, Canny Edge Detection Step by Step in Python — Computer Vision, https://towardsdatascience.com/canny-edge-detection-step-by-step-in-python-computer-vision-b49c3a2d8123
[4] Abhisek Jana, Applying Gaussian Smoothing to an Image using Python from scratch, http://www.adeveloperdiary.com/data-science/computer-vision/applying-gaussian-smoothing-to-an-image-using-python-from-scratch/
[5] Thresholding, Open CV documentation, https://docs.opencv.org/3.4/d4/d86/group__imgproc__filter.html#gaabe8c836e97159a9193fb0b11ac52cf1
