bradautomates

claude-video: Give AI Agents Timestamped Video Evidence

claude-video adds a watch skill that turns video URLs or local files into timestamped visual frames and transcripts for AI agents. It suits people who want an agent to analyze video, with options for local processing or Gemini and transcription services.

PythonMITAI AgentsAgent SkillsComputer Vision
View on GitHub Compare
Stars
18k
Forks
1.9k
Stars, last 10 days
+523

Our take

Health score

36/100

How it is scored
Activity
59
Community
4
Issues
61
Pull requests
21

+523 stars in the last 10 days

  • Fewer contributors than 91% of the projects we track

Overview

claude-video is an agent skill for making video content usable as evidence in AI coding-agent sessions. It can take a video URL or local file and provide timestamped frames and, when available, captions or a speech transcript, so an agent can summarize or answer questions about what happens in the video.

It is intended for Claude Code, Codex, and other local Agent Skills hosts, rather than general chat interfaces. Users can choose a Gemini-based route or a local processing route, depending on their needs for setup, privacy, and available services.

Key Features

  • Accepts video URLs and local files as input.
  • Produces timestamped frames, with options to focus on a time range or select detail level.
  • Uses native captions when available, with optional WhisperX, Groq, or OpenAI transcription fallbacks.
  • Offers a Gemini engine that sends video to Google's video model, or a local engine that extracts frames and audio evidence.
  • Supports installation as a skill or plugin in Claude Code, Codex, and other Agent Skills hosts.
  • Includes setup and diagnostic scripts for checking media dependencies and managing WhisperX installation.

Use Cases

  • A developer can ask an agent to inspect a tutorial or bug-report recording and summarize key events with timestamps.
  • A researcher or analyst can get timestamped visual notes from a public video, using captions or speech transcription when useful.
  • Someone reviewing a local recording can ask questions about visible actions without uploading it to Gemini, by choosing the local engine.
  • Users building workflows around coding agents can add video understanding to an existing Claude Code, Codex, or compatible skills setup.

Getting Started

For Codex, paste this into its message box:

Use $skill-installer to install the watch skill from:
https://github.com/bradautomates/claude-video/tree/main/skills/watch

Then start a new session and ask the agent to use the watch skill. See the README for other installation paths, dependency setup, and usage options.

Considerations

  • The skill is not supported in Claude Chat or Cowork. Cloud-hosted agent sessions can also have different network access and setup persistence from local sessions.
  • The local engine requires Python 3.10+, FFmpeg/ffprobe, and current yt-dlp. YouTube access also needs a supported JavaScript runtime and EJS setup.
  • Gemini processing requires a Google AI Studio API key. Local files and some downloaded URLs are uploaded to Google's Files API when using that engine; the README says uploads are deleted after the answer, or expire within 48 hours if deletion fails.
  • Speech transcription is optional. WhisperX needs a separate environment, model downloads, and substantial disk and memory resources; hosted transcription requires the relevant provider key.
  • Video access depends on source availability, authentication, network permissions, and yt-dlp compatibility. The README recommends using the latest yt-dlp release.

Found this useful?

Share it with someone who would like claude-video.