Using the Polling module in Python

https://github.com/justiniso/polling

I often run into situations when I need a resource that is not yet available. These may be I/O operations or an API requests, but the situation is always the same:

• The resource is currently unavailable
• I expect it to be available in the future
• I don’t know when (or if) it will be available

So I end up writing something like this:

import time
def get_thing():
    ...
while True:
    try:
        my_thing = get_thing()
        break
    except:
        time.sleep(3)

This smells; it’s a sloppy hack. I wanted a reusable and abstract tool to poll, but could not find a stand-alone package. I spent some time thinking about what I wanted that to look like and ended up writing my own — and open-sourcing it. You can install it by:

pip install polling

With polling, you can simply pass a function, a step (time to wait between calls), and a timeout. By default, your function will be called until:

• It returns a truthy value or
• The cumulative execution time exceeds the timeout you specified

You can set other custom options however, such as no timeout with poll_forever=True.

import requests
import polling
# Wait until Google homepage returns 200 status code
polling.poll(
    lambda: requests.get('http://google.com').status_code == 200,
    step=60,
    poll_forever=True
)

There are a number of other options such as the ability to ignore classes of exceptions raised by your function. You can see the full documentation on its repository: https://github.com/justiniso/polling

Using polling with Selenium WebDriver

Polling has been invaluable in my experience with Selenium. The contemporary web is rife with callback-powered JavaScript interactions: elements disappearing and displaying with no way to predict their timing. When you expect an element rendered by JavaScript, a simple find introduces a race condition. Use Polling to locate it reliably:

from selenium import webdriver, NoSuchElementException
driver = webdriver.Firefox()

driver.get('http://google.com')
search_box = polling.poll(
    lambda: driver.find_element_by_id('search'),
    step=0.5,
    timeout=7,
    ignore_exceptions=[NoSuchElementException])
search_box.send_keys('python polling')

The above snippet will go to Google and enter a search term. Normally you are subject to the race between your driver’s query and the script rendering the element in an AJAX callback. But polling will significantly improve the reliability of this script.

Contribute! https://github.com/justiniso/polling