Repository History
3 repositories tagged with GUI Automation
PyAutoGUI: Cross-Platform GUI Automation with Python
PyAutoGUI is an intuitive Python module designed for cross-platform GUI automation, enabling users to programmatically control the mouse and keyboard. It simplifies complex desktop interactions, allowing for tasks like clicking, typing, and taking screenshots across Windows, macOS, and Linux. This powerful library is ideal for automating repetitive tasks and creating bots.
UI-TARS-desktop: The Open-Source Multimodal AI Agent Stack
UI-TARS-desktop is an open-source multimodal AI Agent stack from ByteDance, designed to connect cutting-edge AI models with agent infrastructure. It provides both Agent TARS, a general multimodal AI agent with CLI and Web UI, and UI-TARS Desktop, a native GUI agent for local and remote computer/browser control. This powerful tool aims to enable human-like task completion through rich multimodal capabilities and seamless integration with real-world tools.
OmniParser: A Vision-Based Tool for GUI Agent Screen Parsing
OmniParser is a comprehensive tool developed by Microsoft for parsing user interface screenshots into structured, understandable elements. It significantly enhances the ability of vision-based models, such as GPT-4V, to generate accurate actions grounded in specific regions of a GUI. This project aims to advance pure vision-based GUI agents by providing robust screen parsing capabilities.