How to make a bulk deindex request to Google Webmaster Tool

What is it?

The Google Bulk De-indexer is a tool allowing you to submit one or more URLs to be de-indexed via Google Webmaster Tool in a single request.

Why?

Even though Google Webmaster Tool is powerful, it does not support any way to de-index multiple URLs in a single operation. I needed a way to de-indexing many URLs without having to click through GWT’s UI, one at a time.

Prerequisites

What you need to get this up and running.

  • Ruby (MRI > 1.9.X or JRuby > 1.7.X)
  • Firefox
$ git clone https://github.com/inhang/google-bulk-deindexer.git
$ cd google-bulk-deindexer
$ [sudo] gem install bundler
$ [sudo] bundle install

NOTE: If you’re using RVM/RBenv, you don’t need sudo. Otherwise, you probably do need it.

Installing prerequisites

Credentials

To use this tool with GWT, you need to be able to login to GWT with admin rights to your sites.

You need to add config.json before you can run. For details, you can refer to https://github.com/inhang/google-bulk-deindexer/blob/master/README.md#pre-requisite

How do you use it?

From the command-line

$ ruby deindexer_cmdline.rb file-list-of-urls.txt 
Run from command-line

If you are not comfortable with the command-line, there is a little frontend UI you can use as well.

$ rackup
Starting the UI server

You can go to http://localhost:9292 to see the UI.

Google Bulk Deindexer UI

You can input URLs to deindex, one URL per line.

Inputting URLs

When you click submit, the job will be initiated in the background, and a notification will be sent to those you defined in config.json “mailto” email list along with the status.

Good thing about using the UI is that you can have it running in a remote server, submit a request, close your device, and go about your business. The request will be queued, executed, and you’ll be notified.

Why Firefox? Why Watir/WebDriver?

I wanted to make the tool as unobtrusive as possible to Google Webmaster Tool, mimicking what a human might perform manually. The tool should not abuse Google Webmaster Tool.

Further Information

For more detail, you can refer to the project’s github page at https://github.com/inhang/google-bulk-deindexer

Like what you read? Give David Kim a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.