Find Your Apartment in Berlin Boldly and Quickly with a 10-Minute Winautomation Bot
Apartment hunting can be more painful than falling in love, buying a car, making friends, going to the department of motor vehicles, cleaning, grocery shopping, going to the gyno, or even giving birth, according to the survey conducted by rent.com. In big cities like Berlin, you normally start the search for flats on a number of websites where you have to compete with smart web search engines that might show you search results, which are adapted to their marketing strategies rather than to your very concrete needs. Even if you do find an apartment that suits your criteria, you still have to compete with a hundred other applicants wishing for the same as you. If you are not the lucky selected one, the process goes on and on for you. So, in a fight of a mere mortal with this pitiless system, what are the real tricks that actually do work?
In this article I want to share my experience of how I automated the tedious routine of checking for apartments in internet databases, and as a result, saved my time, nerves, and strength! The automation happens through developing a simple bot that checks an internet website for my specified criteria, saves the matching results in my local database (just an Excel file), and sends me an email when interesting options, that match my criteria arise (see Robotic process automation for more information about bots). Furthermore, my bot is intelligent enough to understand that I want to filter some options that can be understood from the text description of the apartment in an ad. In particular, I don’t want to buy an apartment for investment purposes, rather I actually want to move into the apartment myself. Therefore, I do not want to receive ads that contain words like “rented out” or “investment”.
The ultimate goal of my article is to share with the world how the simple automation of processes can be used for improving your day-to-day life. At the end of the day, why should only companies strive for reduction of resources used for executing processes? In this article I explore in detail the use case of buying an apartment in Berlin. However, these instructions can also be applied for those who are looking to rent an apartment, and of course, for those who want to do it in other cities or countries. Additionally, I’ll be providing some details of apartment searching in Berlin, so don’t miss out ;)
Why Creating Predefined Filters on Search Websites Was Not Good Enough For Me
There already exists a huge amount of websites where you can search for apartments and filter the result data based on your chosen criteria. For example, in Berlin, a lot of people look for an apartment on immonet.de. Let’s imagine that I am searching for an apartment on immonet.de with the following criteria:
Wo suchen Sie? (Eng. Where are you looking?) — Berlin-Wannsee
Option “Kaufen” (Eng. To buy)
Kaufpreis bis (Eng. Price less than) — 350 000
Wohnflache ab (Eng. Square meters starting from) — 70
Zimmer (Eng. Rooms) — 3+
If I click the green button, the website forwards me onto the next page. Important note for the as-is process in this case. Once I enter the search data, the system suggests me to enter my email address so that it could send me the suitable options when new apartments arrive.
My experience with platforms that suggest this, using such predefined filters was not great: the options that they sent to me were not always corresponding to my criteria! In some cases, I received advertisements for flats which cost, say, 500–600 K EUR, although I explicitly stated in the filter that I was looking for a flat which costs less than 350 K EUR. Sometimes I would get lists of flats which would contain the options that I have already viewed. I guess, the platform also needs to promote some concrete options, but that’s not what I am interested in! Also, you normally can’t specify when exactly this email should be sent to you. For example, I would love this email to arrive at the end of my working day so that I won’t be disturbed by it during the course of the day.
After trying uselessly for some time to use the predefined filters of web search engines, I came up with my requirement for the TO-BE searching process of an apartment:
I want that the concrete webpage will be searched for an apartment exactly according to my criteria. The search result should be sent to me via email at the end of my working day. I don’t want to receive advertisements of flats which I have already viewed before.
In the next two Sections, “Prerequisites for Creating a Bot” and “Bot Creating Steps”, I show how I was able to fulfil my requirement, which at the end of the day led me to find the right apartment.
Prerequisites for Creating a Bot:
1. Download and install the Winautomation software. It can be used for free for up to 30 days.
Alternatively, you can use some other RPA-tools like UiPath Community Edition which is completely free for individual developers. All the activities that I describe here can be easily mapped to activities of any standard RPA-tool. Check my other article 5-Minute UiPath Bot: Russian-Speaking Physicians in Berlin where I use UiPath instead of Winautomation. My main observation is in terms of performance they are more or less equal. UiPath might not be very common for those with strong developing background, since it uses workflows and sequences instead of pure code. However, coding in Winautomation also has its drawbacks, e.g., it does not allow to create “If-statements” with multiple conditions. Therefore please bare with my sometimes clumsy coding in the example below.
2. Some very basic understanding of RPA and ability to create at least a simple workflow in Winautomation (you can watch free step-by-step tutorials about creating actions in Winautomation on Youtube).
Bot Creation Steps:
I. Determine the search platform which you would like to be checked automatically.
For example, those who want to search for a flat in Berlin, often use immonet.de or immobilienscout24.de It can be any other website which can yield the results in a structured way (which almost all search websites do), so that it would be possible for the bot to scrape the output data.
II. Enter the desired search criteria and search for the data.
Please enter your search criteria, for example, like Ihave done it in the example above:
III. Wait until the new web-page with the results of your search loads and copy the search link
Once you click Finden (Search), the browser normally forwards you to the web-address that represents a path containing the parameters for your search from the previous step. In our example that would be such link:
Some fields in the link above are understandable, e.g., “toprice = 350000” corresponds to my price wish. Other fields might not be recognizable in a straightforward way. For example, the location parameter “Berlin-Wannsee” was transferred into “locationname=Berlin&acid=&actype=&district=7597&district=39187&district=7894&district=7926&district=7928”.
Sometimes you can tweak the yielded search link to adjust it for your needs. For example, you can add another district in the path, if you know the id of the district (on some websites the ID would simply be the text name of the district). This would give you a nice filtering link which would show you results by two districts of your choice, and such search options might not even be available in the first screen of the search website.
IV. Create the following sequence in Winautomation:
The code consists of 3 regions. I dive into details of each region in corresponding subsections below.
1 — Region “Work with a Browser”
Firstly, tell the bot to launch a new browser and navigate to the link that you copied from the browser on Step III (Line 2 of the code). In this example, I used Internet Explorer (IE) for the browser type. However, other browsers, such as Google Chrome or Firefox, can also be used.
When the bot executes this action, the page with the results of the search will be opened. It will look like something like this:
Secondly, use the action “Extract Data from Web Page” to extract structured information from the web-page that was opened (Line 3). The properties of my action “Extract Data from Web Page” look as follows:
In particular, I extract data from the page in the form of a 3-Column Table that I store as a variable %DataFromWebPage%. In order to extract data from the web page, I have utilized the Extract Data activity. If you need further help with this activity, please watch the tutorials on Youtube. The corresponding values that I would like to store in my internal database look as follows:
In particular, Value #1 — the title of the ad, Value #2 — the price of the apartment, and Value #3 — the unique URL of the ad which would also serve as a unique ID in my internal database.
NB: I explain later in the next section why I want to keep the title of the ad, while it might seem reasonable to only keep the unique URL and maybe the price.
2 — Region “Process Extracted Data Table”
In this region, I tell the bot to open an excel file which serves as my internal database (Line 7) and read the data which was already stored in this Excel file before (Lines 10–11). In Lines 12–49, the bot checks if the data extracted from the web-page (Region 1) corresponds to my predefined conditions and should be added to the excel file. If that’s the case, the bot adds the corresponding entries in Lines 36–44.
In Lines 17–25 the bot parses the text for the presence of certain keywords. I already mentioned earlier in Region 1 that I want to keep the title of the ad while scraping the data from the website because later I will do some parsing on the text and exclude some options which are not of interest to me. For example, people often sell houses, marketed as an investment, while someone continues to live there. I don’t want such a flat since I want to live in the apartment myself. Therefore, I don’t want the ad to contain such words (in German) as “investition” (see Line 17), “kapital” (see Line 21), or “vermietet” (see Line 25) — they all imply that the place is rented out and buying would most likely be for the purpose of investing (you can’t just simply kick people out of the places where they live, at least not in Germany!).
NB: You might have your own desires for a flat which you can map to the text to be excluded from consideration, apart from the investment purpose. Maybe you want to exclude such options as “Old or Modern Building” etc.
If an entry from the webpage is unique and does not contain the words to be excluded, the bot writes to the Excel file (Line 41) and moves further in the loop.
3 — Region “Specify and Send Email”
Since the work of the bot is done in the background without my involvement at all, as the last step, I want to be notified by email if my bot found options matching my criteria.
If the bot found some unique relevant entries that were added to the Excel file, it sends me an email which would provide the most important information about these options (Line 57):
Otherwise, I at least want to know that the bot worked today even though has not found anything (Line 59):
If you don’t want to receive a “negative” result, just simply omit this action.
V. Add a schedule to the created Bot:
Now the workflow for the bot is finished, but it is important to determine when the bot will be executed. In particular, Winautomation offers an option of adding schedules on processes. Right click on the relevant process in your Winautomation console -> Add Schedule on Process -> Schedule.
The window “Schedule Properties” will open where you can customize the options as you wish:
Result
Now, each day at 18:00 PM the bot executes the steps which I described earlier. As a result, I get an email which looks like this one:
Alternatively, if the bot hasn’t found anything, I get such an email:
My involvement in this process is zero, I basically only need to read emails when they come, be happy if I have some potential options and proactively request an appointment for viewing the flat.
To increase the scope of the work of the bot, I created several such bots for several web search platforms, and after some weeks of searching, I found the flat that I was interested in! The criteria mentioned in the examples above do not necessarily reflect the exact criteria I used, but the steps were the same. Therefore, this is a story where automation of the daily process really brought a real and pleasant result.
If you are interested in other cases of automation of processes that can improve your daily life, check my other article 5-Minute UiPath Bot: Russian-Speaking Physicians in Berlin. Do you have your own examples of how you used Robotic Process Automation (RPA) solutions to make your life easier? Share it in the comments, your experience would be very welcome!
