Improving Accessibility with Python Keyboard and Mouse Automation

Boosting productivity with PyAutoGUI

PyAutoGUI is a Python library that allows users to automate GUI interactions on a desktop computer. It’s a very powerful library that can perform tasks like automating mouse movements, keystrokes, and clicks. In this article, we’ll review how to use PyAutoGUI and provide some example code.

This module is not preloaded with python. So to install it run the following command:

pip install pyautogui

Mouse automation

Let's go back to our thirst grade social science lessons where we used to plot places on the map with the help of the latitudes and longitudes on the edge of the map. Similarly, we can point, click or drag on our monitor automatically just by specifying the correct coordinates of our screen. PyAutoGUI considers our screen a graph sheet and executes the commands on the given coordinates. It varies with the screen as different systems have different screen sizes.

So technically you would have to know the size of the screen. You can find your screen size by

import pyautogui
print(pyautogui.size())

Let's automate some of the commonly used mouse functions:

1. Clicking

import pyautogui
pyautogui.click(500, 500)

2. Pointing

import pyautogui
pyautogui.moveTo(500, 200)

3. Dragging

import pyautogui
pyautogui.moveTo(1000, 1000)
pyautogui.dragRel(100, 0)
#drags form (1000, 1000) to (100,0)

4. Scrolling

import pyautogui
pyautogui.scroll(200)

Keyboard automation

You can also automate all your keyboard functions using the same library. The main function of the keyboard is to type, as a developer I use it more for copy-pasting code from online using shortcut keys. We can easily automate both of these functions.

1. Typing

import pyautogui
pyautogui.write("hello world...!!")

2. Shortcut Keys

It can be done in the same way as we type using typewrite just by specifying the keys in square brackets.

import pyautogui
pyautogui.write(["ctrl", "c"])

It can also be done using the hotkey function

import pyautogui
pyautogui.hotkey("ctrlleft", "a")

You can also use the press function()

import pyautogui
pyautogui.press('enter')
pyautogui.press('left', presses=3)

Conclusion

PyAutoGUI is a powerful Python library that allows us to automate GUI interactions on a desktop computer. In this article, we covered some of the basics of PyAutoGUI, including moving the mouse, clicking the mouse, typing and keyboard shortcuts, and taking screenshots. With PyAutoGUI, we can automate repetitive tasks and save time and effort in our daily work.