How to project an image in perspective view of a background image — OpenCV, Python
We all have come across and used web apps where we can insert our pictures into other background images like buildings, hoardings etc. As a kid, I always wondered about the process of applying an image to a perspective view of a building. While browsing the internet, I came across the web app, and was driven towards making its program. With a prior knowledge of OpenCV, I started working on the project and it was successfully done in about 3.5 hours!
Before we dive in, I would like to share with you the workflow of the project.
Here is a step by step guide to code the python program-
Step-1: We need to find the coordinates of corners from the first image(portrait) and the corresponding coordinates of the points from the second image(building).
a. Getting the coordinates of corners from the first image
#Storing width and height of first image in w1 and h1
h1,w1 = dp.shape[:2]
b. Getting the four coordinates from the second image using a mouse click event
# Mouse callback function
# If event is Left Button Click then store the coordinate in the lists
if event == cv2.EVENT_LBUTTONUP:
# Reading the two images and storing it in variables building and dp
building = cv2.imread('building.jpeg')
dp = cv2.imread('dp.jpg')# Defing a window named 'image'
k = cv2.waitKey(20) & 0xFF
if k == 27:
Note: After selecting the four coordinates in the second image press Esc key.
Step-2: Now that we have the coordinates of the pictures. We will calculate the homography matrix using the cv2.findHomography() function.
h, mask = cv2.findHomography(pts1, pts2, cv2.RANSAC,5.0)
We get the following matrix-
Step-3: Now we will use cv2.warpPerspective() function which will take the inputs image1(dp), homography matrix(h) and width and height of the second image, and will give the following output.
im1Reg = cv2.warpPerspective(dp, h, (width, height))
Step-4: We will make a mask using np.zeros() of the size of the second image. This will be as shown in the figure below
Step-5: We will now fill the mask with white area using cv2.fillConvexPoly() function which will return the output as shown
Step-6: Now we will invert the obtained mask using cv2.bitwise_not() function.
Step-7: We will then use cv2.bitwise_and() to get the following output.
Step-8: We will use cv2.bitwise_or() function to get the final output.
Step-9: Finally, we will save this image using cv2.imshow() function.
Recommended: Do checkout the workflow of the project.
Here is the GitHub link and a short demo of my project.
P.S.- If you like the project, star the repo :))
Here are some links to the concepts used in this blog:
- If you are new to Computer Vision and want to know the basic stuff about OpenCV. Getting started with Opencv and Python will be a good read.
- To learn more about Perspective Transformation, go through this example Project — 4 Point OpenCV getPerspective Transform Example
- A great source to learn about Homography (with examples in Python, C++) — Homography Examples using OpenCV
Let me know if you have any suggestions/feedback!