Automatic Smart Collage Generation for Property Images | Collager

The whole is greater than sum of its parts

— Aristotle

In visual art, a collage is assemblage of different pieces of images in a unique way which combined to give whole new perspective than individual images. Although invention of collage can be traced back to 300 BC, this art form became famous in beginning of 20th century.

NoBroker is the World’s largest brokerage free C2C platform for house hunting. We deal with thousands of new property images per day. Images form a very important part of our listing. Visual content forms one of the primary deciding factors in user’s interaction on listings.

We have seen that properties with good photos receive five times more interactions than those with poor or no photos. Hence good quality images play an important role. A good collage can convey holistic information better than individual images. This motivated us to build a system that creates stunning collages with property images. This in turn can be used as feature picture of the property in listing, marketing content, recommendations etc.

Collager is our in-house smart collage making engine. With the power of ML and Image Processing, we assemble images to give eye-catching holistic view of the property. In further sections we are going to elaborate the components of this engine.

Sample Collages

Before jumping into technicalities of Collager, lets have a look on collages made by this system.

Some specifications of the Collager:

1. Collager picks only those images which are informative and aesthetically appealing to tailor a collage done by ML models
2. Collager is made such that at least three images are needed in a property for collage to be made
3. Collager organizes the image layout in such a way that the largest/feature image in a collage is the best looking image in the collection
4. For the same number of images, there are a number variations of collages possible. In fact, it has 120 variations or templates of collages!
5. Collages can be both portrait or landscape
6. Collages have been made in such as way that it not only catch attention of users but also convey best thing about the property through collage.

Now let’s understand workflows and the components of this system and brief idea about how we built these components.

Collager Workflow

Images are uploaded in large numbers on our platform which are processed by Image Intelligence system Iris. To know about this system please refer here. We get complete understanding like — room type, nima_score, depth_score (perspective depth) etc. of the image. We extract these data from Image Meta Database which are used in Filtering Engine for screening images. Images of poor quality and low resolution are filtered and screened images are passed on to Collager Maker to make collages using Template Engine. Lets dive deeper into various components of the Collager system.

Diving Deeper

Collager has four main components:

Data Extraction: Tens of thousands of property images are uploaded per day on our platform. Each image is passed through Iris which uploads images in the bucket and update meta info of the image in Image Meta Database. Data Extraction module pulls these meta data containing nima_score, depth_score and room type of the image. These data are passed to Filtering Engine which we are going to see next.

Filtering Engine: Here, we calculate goodness score for each image which is defined as weighted sum of nima_score, depth_score and room_score where nima_score is Neural Image Assessment Score of the image, depth_score is perspective depth in the image predicted by ML model and room_score is a score given to room_type tagged by room classification model. For example, if there are four classes of room viz. bedroom, kitchen, balcony and outdoor, room_scores are 3, 2, 1, 4 respectively based on preference we want to give to various room types. Higher the preference, greater is the score. The goodness score reflects how much attractive and stunning given image is. This goodness score is used later in this component.

Filtering Engine is a multi-step filtering process which uses nima_score and depth_score to accept or reject a image to be included into collage. Images with nima_score and depth_score greater than some threshold are qualified for collage making. This filtering process runs for each image per property. If property has at least three images after filtering then it is qualified for collage making else not. Images for a qualified property are then sorted in descending order of their goodness score and passed to Collage Maker. Meta info about rejected properties are updated in the Collager Database which contains reasons for rejection to be used in retrospective analysis.

Collage Maker: Images that have been screened from the Filtering Engine are passed on to the Collage Maker module. Here each image is pasted on collage template decided by Template Engine. We also add nobroker logo on the collage and watermark on each image in the collage. In this way, collage using qualified images are made and then collage is uploaded back to bucket along with meta info updated in the Collager Database.

Template Engine: This is the Brain of the Collager. It decides collage template given number and orientation of images to be used in collage. Suppose there are five images to be used in collage which are sorted via their goodness score and first two images are landscape and last three images are portrait, then orientation of images is ‘llppp’ where ‘l’ denotes landscape and ‘p’ denotes portrait.

Collage template is a nested dictionary with number of images and orientation of images as keys and size of collage in pixels, coordinates (x, y) and final size of each image as values which are used while pasting each image in the collage. These coordinates and final size are decided in such a way that whole collage looks attractive and also minimum loss happen while cropping individual image before pasting in the collage. Laws of perspective of visual art played a crucial role in making the Template Engine.

Collager will form an integral component in a number of our property previews — in recommendations, ads, push notifications etc. With one image being to convey more about the property than a single image, we are able to capture greater attention and interest level on our listings. Hence Collager affects the needle mover of our business, which is C2C interactions.

Since the beginning we have been streamlining real estate by algorithmically adapting to a number of inefficiencies in Real Estate, right from brokerage to pricing asymmetries. We post about a number of our efforts in technology in the blog publication. Follow us to get notified. Also we are constantly looking out for people who have grit to break and make stuff.

Wanna join us in eternal journey of learning and innovation? Come on, we are hiring:)