UFO: Automate Desktop Tasks Across Devices with AI Agents

Overview

UFO is Microsoft's Python project for AI-driven desktop automation. It brings together UFO², a Windows-focused agent that interacts with apps through GUI and API actions, and UFO³ Galaxy, which coordinates tasks across multiple device agents.

It is aimed at people building automated workflows that involve desktop applications or several devices. UFO² is the more direct fit for individual Windows tasks; Galaxy is intended for workflows that benefit from task dependencies, device selection, and coordinated execution. Galaxy is described as being in active development, while UFO² is presented as the stable, long-term-support option.

Key Features

• Automates Windows desktop tasks using GUI interaction and API actions through UFO².
• Decomposes multi-step requests into a dependency-aware task DAG in Galaxy.
• Assigns work to devices based on capabilities and task requirements.
• Supports asynchronous and parallel execution of independent tasks.
• Adapts Galaxy task graphs using execution results and feedback.
• Provides a WebSocket-based Agent Interaction Protocol for agent coordination.
• Supports device agents across Windows, Linux, and Android, with additional platforms described as planned.
• Allows UFO² to operate standalone or as a Windows device agent in Galaxy.

Use Cases

• A Windows user automating repetitive work across desktop applications can start with UFO².
• A developer coordinating tasks between Windows and Linux devices can use Galaxy to assign and track work across agents.
• Teams with multi-step workflows can use Galaxy when tasks have dependencies or independent steps that can run in parallel.
• Agent developers can build device-specific agents and extend their capabilities with MCP integrations.

Getting Started

The repository README provides separate setup instructions for Galaxy and UFO². Both use requirements.txt and require LLM API configuration. For Windows automation, the README's quick start runs python -m ufo --task <task_name> after installation and configuration. See the repository README and documentation for setup details.

Alternatives

• Open-Interface: Open-Interface automates computer tasks through keyboard, mouse, and screenshots, while UFO also coordinates agents and assigns work across devices.

Considerations

UFO² is focused on Windows desktop automation, so it is not a general-purpose cross-platform desktop agent by itself. Galaxy supports broader device orchestration but is identified in the README as under active development, with advanced device support and monitoring still in progress. Both paths require configuring an LLM provider; Galaxy also involves setting up device agents and orchestration configuration.