# CoTracker: A Powerful Model for Tracking Any Point on a Video

This repository profile is provided by osrepos.com, an open source repository discovery platform.

Source: osrepos.com
Repository profile: https://osrepos.com/repo/facebookresearch-co-tracker
Generated for open source discovery and AI-assisted research.

CoTracker is a state-of-the-art model developed by Facebook AI Research and the University of Oxford, designed for tracking any point (pixel) across video sequences. This transformer-based solution offers fast, accurate, and quasi-dense point tracking capabilities. It is an invaluable tool for researchers and developers in computer vision, enabling precise analysis of motion in videos.

GitHub: https://github.com/facebookresearch/co-tracker
OSRepos URL: https://osrepos.com/repo/facebookresearch-co-tracker

## Summary

CoTracker is a state-of-the-art model developed by Facebook AI Research and the University of Oxford, designed for tracking any point (pixel) across video sequences. This transformer-based solution offers fast, accurate, and quasi-dense point tracking capabilities. It is an invaluable tool for researchers and developers in computer vision, enabling precise analysis of motion in videos.

## Topics

- optical-flow
- point-tracking
- track-anything
- Jupyter Notebook
- Computer Vision
- Deep Learning
- Video Analysis
- AI/ML

## Repository Information

Last analyzed by OSRepos: Sat Apr 11 2026 12:21:20 GMT+0100 (Western European Summer Time)
Detail views: 6
GitHub clicks: 9

## Safety Notice

OSRepos shares public repositories for knowledge and discovery only. Review source code, dependencies, licenses, and security implications before running or installing anything.

## Content

## Introduction

CoTracker, developed by Meta AI Research and the University of Oxford, is a cutting-edge model designed to track any point or pixel within a video. Leveraging a fast transformer-based architecture, CoTracker brings the benefits of optical flow to point tracking, offering robust and accurate performance.

Key features of CoTracker include:

*   Tracking any individual pixel in a video.
*   Tracking a quasi-dense set of pixels simultaneously.
*   Flexibility to select points manually or sample them on a grid in any video frame.

The project has seen continuous development, with significant updates including CoTracker3, which offers simpler and better point tracking through pseudo-labelling real videos, and the release of the Kubric Dataset for enhanced training. CoTracker has also been instrumental in other projects, such as VGGSfM, a fully differentiable SfM framework.

## Installation

CoTracker can be easily integrated into your projects. A GPU is strongly recommended for optimal performance.

### Via PyTorch Hub

The easiest way to get started is by loading a pretrained model from `torch.hub`.

**Offline mode:**

bash
pip install imageio[ffmpeg]


python
import torch
import imageio.v3 as iio

url = 'https://github.com/facebookresearch/co-tracker/raw/refs/heads/main/assets/apple.mp4'
frames = iio.imread(url, plugin="FFMPEG")

device = 'cuda'
grid_size = 10
video = torch.tensor(frames).permute(0, 3, 1, 2)[None].float().to(device)

cotracker = torch.hub.load("facebookresearch/co-tracker", "cotracker3_offline").to(device)
pred_tracks, pred_visibility = cotracker(video, grid_size=grid_size)


**Online mode:**

python
import torch
import imageio.v3 as iio

url = 'https://github.com/facebookresearch/co-tracker/raw/refs/heads/main/assets/apple.mp4'
frames = iio.imread(url, plugin="FFMPEG")

device = 'cuda'
grid_size = 10
video = torch.tensor(frames).permute(0, 3, 1, 2)[None].float().to(device)

cotracker = torch.hub.load("facebookresearch/co-tracker", "cotracker3_online").to(device)
cotracker(video_chunk=video, is_first_step=True, grid_size=grid_size)

for ind in range(0, video.shape[1] - cotracker.step, cotracker.step):
    pred_tracks, pred_visibility = cotracker(
        video_chunk=video[:, ind : ind + cotracker.step * 2]
    )


### Development Version

To install CoTracker from the GitHub repository, which is ideal for running local demos or for evaluation and training:

bash
git clone https://github.com/facebookresearch/co-tracker
cd co-tracker
pip install -e .
pip install matplotlib flow_vis tqdm tensorboard


## Examples

CoTracker provides various ways to interact with its powerful tracking capabilities.

### Visualize Predicted Tracks

After installation, you can visualize the tracks with a simple script:

python
from cotracker.utils.visualizer import Visualizer

vis = Visualizer(save_dir="./saved_videos", pad_value=120, linewidth=3)
vis.visualize(video, pred_tracks, pred_visibility)


### Interactive Demos

*   **Hugging Face Space:** Explore the interactive demo on the [facebook/cotracker Hugging Face Space](https://huggingface.co/spaces/facebook/cotracker "Hugging Face Space" target="_blank").
*   **Google Colab:** Run the notebook directly in [Google Colab](https://colab.research.google.com/github/facebookresearch/co-tracker/blob/master/notebooks/demo.ipynb "Google Colab Demo" target="_blank").
*   **Local Gradio Demo:** Run the Gradio demo locally by installing requirements (`pip install -r gradio_demo/requirements.txt`) and executing `python -m gradio_demo.app`.

### Command Line Demos

*   **Offline Demo:** `python demo.py --grid_size 10` (results saved to `./saved_videos/demo.mp4`).
*   **Online Demo:** `python online_demo.py`.

## Why Use CoTracker?

CoTracker stands out as a robust solution for point tracking due to several compelling reasons:

*   **State-of-the-Art Performance:** CoTracker3 achieves impressive results on benchmarks like TAP-Vid, demonstrating superior accuracy compared to previous versions and other models.
*   **Versatility:** It can track any pixel, a quasi-dense set of pixels, and supports both manual and grid-based point selection.
*   **Efficiency:** The transformer-based architecture ensures fast processing, and the online mode offers memory-efficient tracking for longer videos.
*   **Continuous Improvement:** The project is actively maintained with regular updates, including new models, datasets, and features.
*   **Developer-Friendly:** With PyTorch Hub integration, clear installation instructions, and various demo options, CoTracker is accessible for both quick experimentation and in-depth development.

## Links

*   **GitHub Repository:** [https://github.com/facebookresearch/co-tracker](https://github.com/facebookresearch/co-tracker "CoTracker GitHub" target="_blank")
*   **Project Page:** [https://cotracker3.github.io/](https://cotracker3.github.io/ "CoTracker Project Page" target="_blank")
*   **Paper #1 (ECCV 2024):** [https://arxiv.org/abs/2307.07635](https://arxiv.org/abs/2307.07635 "CoTracker ECCV Paper" target="_blank")
*   **Paper #2 (arXiv 2024):** [https://arxiv.org/abs/2410.11831](https://arxiv.org/abs/2410.11831 "CoTracker3 arXiv Paper" target="_blank")
*   **X Thread:** [https://twitter.com/n_karaev/status/1742638906355470772](https://twitter.com/n_karaev/status/1742638906355470772 "CoTracker X Thread" target="_blank")
*   **Colab Demo:** [https://colab.research.google.com/github/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb](https://colab.research.google.com/github/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb "CoTracker Colab Demo" target="_blank")
*   **Hugging Face Space:** [https://huggingface.co/spaces/facebook/cotracker](https://huggingface.co/spaces/facebook/cotracker "CoTracker Hugging Face Space" target="_blank")