Skill Recorder: Turn Screen Recordings into AI Agent Skills

Introduction

Skill Recorder is an innovative desktop application developed by Microsoft that transforms your real-world work sessions into actionable AI agent skills. By recording your screen activity, it intelligently reconstructs your actions into a clear intent and a sequence of steps, enabling the creation of reusable automations. This tool is designed to empower users to teach AI agents new tasks efficiently, bridging the gap between human interaction and AI automation.

Why Use It and Key Benefits

Skill Recorder offers a unique approach to automation by learning directly from your actions. Instead of manual programming, you simply perform a task once, and the application, powered by GitHub Copilot CLI, translates it into a structured procedure. Key benefits include:

Effortless Skill Creation: Record any task, from form submissions to complex workflows, and turn it into a reusable skill or automation.
AI-Powered Analysis: Utilizes GitHub Copilot CLI to understand the intent and steps of your recorded session.
Agent Native Tools: Prioritizes using an agent's native tools, like gh CLI or web_fetch, over replaying UI clicks, leading to more robust and adaptable automations.
Generalization: Learns from a single example to generalize tasks, allowing an agent to perform similar actions in various contexts.
Integration: Builds skills compatible with Microsoft Scout, Microsoft Copilot Cowork, or Copilot Studio.

Installation

Skill Recorder is distributed as a source release, meaning a single command downloads a pinned Node.js runtime, builds the exact release commit, and adds the application to your system. You will need a GitHub account with Copilot access. The Copilot CLI is bundled with the app. macOS is the primary target, with Windows 11 (x64 and ARM64) also supported.

To install, visit the latest release page and copy the command for your platform. The commands below are patterns, where <40-character-release-commit> must be replaced with the specific commit from the release page.

macOS / Ubuntu

commit="<40-character-release-commit>"; curl -fsSL "https://raw.githubusercontent.com/microsoft/skill-recorder/$commit/install.sh" | SKILL_RECORDER_COMMIT="$commit" bash

To keep the app running after the terminal closes, add SKILL_RECORDER_DETACHED=1 after the pipe:

commit="<40-character-release-commit>"; curl -fsSL "https://raw.githubusercontent.com/microsoft/skill-recorder/$commit/install.sh" | SKILL_RECORDER_COMMIT="$commit" SKILL_RECORDER_DETACHED=1 bash

Windows (PowerShell)

$commit="<40-character-release-commit>"; $env:SKILL_RECORDER_COMMIT=$commit; irm "https://raw.githubusercontent.com/microsoft/skill-recorder/$commit/install.ps1" | iex

After installation, grant Screen Recording permission on first launch (macOS). Then, you can start recording, analyzing, and creating skills. The first time you analyze, Skill Recorder will prompt you to sign in to Copilot if you haven't already.

Important Security Note: Do not record, type, paste, or narrate passwords, tokens, API keys, or other confidential information. Choosing "Analyze" sends recording data to GitHub's cloud. Skill Recorder reminds you before every recording.

Examples

The process of creating a skill with Skill Recorder is straightforward:

Record: Start recording your task with a click or ??R / Ctrl+Shift+R. Skill Recorder captures your screen and activity locally.
Control: A small, always-on-top bar allows you to manage the recording, including microphone state.
Analyze: Click "Analyze," and GitHub Copilot reconstructs your actions into an overall intent and an ordered list of steps. You can review and edit this analysis.
Create: From an approved analysis, generate a reusable Skill (a SKILL.md procedure) and/or a scheduled Automation.

Skill Recorder captures window tracking, browser URLs, screen video snapshots, short clipboard previews, and optional on-device narration transcription. All data remains local until you explicitly choose "Analyze," at which point relevant event timelines, screen images, and narration text are sent to GitHub's cloud for Copilot processing.