Dable Blog
Jul 10 · 2 min read

The robots.txt file is a text file that tells web robots which pages on your site to crawl or not.

Step 1. Open Notepad

▶ Open a default text editor program in your computer, e.g. Notepad.

​▶ Copy and paste the following to the Notepad.

User-agent: AdsBot-Google-Mobile
User-agent: AdsBot-Google
User-agent: Googlebot
Allow: /

※ This means you are allowing Googlebot (desktop, mobile, and Search) to crawl your site.

Step 2. Save File and Upload

▶ After creating the robots.txt file, please save the file under the name of ‘robots.txt’.

▶ Upload the file to root directory (root folder) of your site.

Root directory is like a ‘parent(root) folder’ in folder structure.

Q. In a root folder named ‘Dable,’ we have three sub folders — Company A, Company B, and Company C. In this case, what is the root directory of this site?

A. The answer is — ‘Dable’.

▶ Often, root directory has public_html, html, www, wwwroot, and htdocs as its name.

If you cannot find any folder with these names,
please look for a folder that has an index file (index.php, index.html, default.html) in it.
Then upload the robots.txt file in that folder (note that it could be an html folder under www). You’re all set!​

Step 3. Check and Confirm

▶ For example, if your website address is www.dable.io and you can see all the things you have entered in the Notepad when you type www.dable.io/robots.txt in the address bar, then you can confirm your robots.txt file has been successfully uploaded.

※ Please note that robots.txt file should be placed in www.dable.io/robots.txt, not in www.dable.io/new/robots.txt.

※ There should be only one robots.txt file for each site.


We hope these tips will be helpful in enjoying a successful advertisement experience through Dable! We will always strive to provide better information to help you increase profits.

If you wish to know more about our services or have any suggestions, please contact ad@dable.io.

Dable Blog

№1 Content Discovery Platform — Personalized Recommendation & Native Advertising

Dable Blog

Written by

№1 Content Discovery Platform — Personalized Recommendation & Native Advertising

Dable Blog

№1 Content Discovery Platform — Personalized Recommendation & Native Advertising

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade