PyAutoGUI: Cross-Platform GUI Automation with Python

Introduction

PyAutoGUI is a powerful, cross-platform Python module that simplifies GUI automation for human beings. It allows you to programmatically control the mouse and keyboard, making it an excellent tool for automating repetitive tasks, creating bots, or performing desktop interactions without direct human input. Whether you need to click buttons, type text, or capture screenshots, PyAutoGUI provides a straightforward API to achieve these actions across Windows, macOS, and Linux.

Installation

Installing PyAutoGUI is simple using pip. However, note that there are some operating system-specific dependencies.

To install PyAutoGUI:

pip install pyautogui

Dependencies:

• Windows: No additional dependencies.
• macOS: Requires pyobjc-core and pyobjc modules.
• Linux: Needs python3-xlib (or python-xlib for Python 2).
• All Platforms: Pillow needs to be installed for image-related features. On Linux, additional libraries might be required for Pillow's PNG/JPEG support.

Examples

PyAutoGUI offers a wide range of functions for controlling your GUI. Here are some common use cases:

Keyboard and Mouse Control

Control mouse movement, clicks, and keyboard input. The origin (0,0) is the top-left corner of the screen.

import pyautogui

screenWidth, screenHeight = pyautogui.size() # Returns two integers, the width and height of the screen.
currentMouseX, currentMouseY = pyautogui.position() # Returns two integers, the x and y of the mouse cursor's current position.
pyautogui.moveTo(100, 150) # Move the mouse to the x, y coordinates 100, 150.
pyautogui.click() # Click the mouse at its current location.
pyautogui.click(200, 220) # Click the mouse at the x, y coordinates 200, 220.
pyautogui.move(None, 10)  # Move mouse 10 pixels down, relative to its current position.
pyautogui.doubleClick() # Double click the mouse at the current location.
pyautogui.moveTo(500, 500, duration=2, tween=pyautogui.easeInOutQuad) # Use tweening/easing function to move mouse over 2 seconds.
pyautogui.write('Hello world!', interval=0.25)  # Type with quarter-second pause in between each key.
pyautogui.press('esc') # Simulate pressing the Escape key.
pyautogui.keyDown('shift')
pyautogui.write(['left', 'left', 'left', 'left', 'left', 'left'])
pyautogui.keyUp('shift')
pyautogui.hotkey('ctrl', 'c')

Display Message Boxes

PyAutoGUI can display various types of message boxes, useful for user interaction or debugging.

import pyautogui

pyautogui.alert('This is an alert box.')
pyautogui.confirm('Shall I proceed?')
pyautogui.confirm('Enter option.', buttons=['A', 'B', 'C'])
pyautogui.prompt('What is your name?')
pyautogui.password('Enter password (text will be hidden)')

Screenshot Functions

Capture screenshots and even locate images on the screen.

import pyautogui

im1 = pyautogui.screenshot()
im1.save('my_screenshot.png')
im2 = pyautogui.screenshot('my_screenshot2.png')

# Locate an image on the screen
button7location = pyautogui.locateOnScreen('button.png') # returns (left, top, width, height) of matching region
# Example output: (1416, 562, 50, 41)
buttonx, buttony = pyautogui.center(button7location)
# Example output: (1441, 582)
pyautogui.click(buttonx, buttony)  # clicks the center of where the button was found

# Locate and click the center of an image directly
buttonx, buttony = pyautogui.locateCenterOnScreen('button.png') # returns (x, y) of matching region
# Example output: (1441, 582)
pyautogui.click(buttonx, buttony)  # clicks the center of where the button was found

Why Use PyAutoGUI?

PyAutoGUI excels at abstracting the complexities of operating system-specific GUI automation APIs. Instead of dealing with the intricate details of Windows API, macOS Cocoa API, or Linux X11, developers can use a single, consistent Python interface. This cross-platform compatibility and ease of use make PyAutoGUI an ideal choice for:

• Automating repetitive tasks: From filling out forms to navigating applications.
• Testing GUI applications: Creating automated test scripts for user interfaces.
• Creating simple bots: For games or other desktop applications.
• Accessibility tools: Assisting users with specific needs by automating interactions.

Its design focuses on being "for human beings," meaning its functions are intuitive and mimic natural user interactions.

Links

• Full Documentation: https://pyautogui.readthedocs.org
• Source Code (GitHub): https://github.com/asweigart/pyautogui
• Simplified Chinese Documentation: https://github.com/asweigart/pyautogui/blob/master/docs/simplified-chinese.ipynb

Support the Project

If you find PyAutoGUI helpful and wish to support its ongoing development, consider donating to its creator on Patreon: https://www.patreon.com/AlSweigart