Automatically filling multiple responses into a Google Form with Selenium and Python
A Google Form is one of the most widely used and easiest ways to get data from lots of people. Just make a form, send it out and the results magically come into a spreadsheet. It’s that simple!
Sometimes though, you may find yourself in a position where you want to fill the same form multiple times. Whatever your reasons may be, doing so is very easy! But use this information wisely and only for educational purposes :P
If you want a demo Google Form to follow along with, you can use this one https://forms.gle/PxHxoVDDsgvmftg98
This tutorial is based on but isn’t limited to Google Forms. You can modify it to suit any website and all your scraping needs.
In order to automate this task, we will be using Selenium and Python. Minimal experience with Python and HTML is recommended but not necessary. I’ll be using Google Chrome as my browser of choice here, but you can of course use any other.
Step 1: Install the Webdriver
Selenium automates the browser movement and we need a driver for it to accomplish that. First, we need to find our browser’s version number. This is to make sure we install the correct webdriver.
For Google Chrome, click on the 3 dots at the top right. Then click on Help and finally About Google Chrome.
You should see the version number right in the middle of the tab that opens up!
Then head to https://chromedriver.chromium.org/downloads and choose the download corresponding to your version number and operating system.
If you’re using a different browser, simply Google and you’ll be able to find the correct webdriver.
Step 2: Install Selenium
Navigate to your preferred directory and create a virtual environment. We will be using Python 3 in this tutorial. You can learn more about virtual environments at https://docs.python.org/3/tutorial/venv.html.
Install Selenium using the command pip install selenium
Step 3: Configuring Selenium Webdriver
Create a new file. To use Selenium, first, we have to import it. Then we can add the options that want. A very useful option is incognito. This prevents the browser from storing any history, cookies, or cache or using any of these which were previously stored.
If you're inputting a lot of data, using a headless browser might be useful. A headless browser runs in the background without a GUI. The --headless and disable-gpu options accomplish these. You can uncomment them if you want to run headless.
Finally, we provide the path to the webdriver that we downloaded in Step 1. Replace the path with your own. If you’re using Windows, remember to add the .exe extension at the end.
Step 4: Finding the elements in the Webpage
Now, we link the webpage you want to open. Add the line browser.get("https://forms.gle/FoAoauz53Xy7A4n68") to your code. Replace the link with your target website.
Next, open up the website normally and press Ctrl+Shift+c. This should open up a tab on the right and will show you the HTML code for whichever element you hover on.
Find the elements that you want to use (Textboxes, checkboxes, radio buttons, etc.) Each one of these elements is grouped according to its type with a common class value. Usually, this is the first value in the class field.
On different websites, you might find an id value. This is better to use as it makes sure you choose the correct element.
An easier option that is likely to work for most elements, regardless of the website is XPath. Select the element you would like, right-click on it and then choose Copy -> Copy XPath. Use can use this copied value in the next step.
Step 5: Interacting with the elements
find_elements_by_class_name returns a list of all elements with that class. If you want only the first occurrence of that class name or id value, you can also use find_element_by_class_name or find_element_by_id (in the singular). In the list returned by the former, you can access individual elements using indexing. If you are using the XPath of the element, you can use find_element_by_xpath.
When dealing with textboxes, the most common thing you may want to do is adding text to them. You can do this by the .send_keys() function and pass the string that you want to be typed as the argument.
To click on a button, checkbox, or radio button use the .click() function.
And that's it! That’s pretty much all that’s needed to handle a Google Form. Different question types or multiple pages require a bit of deft maneuvering but can be tackled with ease.
When building a script for a new website, I’d recommend using a Jupyter notebook or an editor that allows you to run individual cells/lines of code. That way you can modify certain parts of the script without running everything again.
By using loops and logic, the usage can be expanded for all testing, scraping, and automation needs. Use wisely!
If you want to go deeper into Selenium and explore more ways of finding and interacting with elements, you can refer to the documentation at https://selenium-python.readthedocs.io/
