vidgear: High-Performance Cross-Platform Video Processing Framework in Python

Introduction

vidgear is a high-performance, cross-platform Python framework that provides a comprehensive, one-stop solution for building complex real-time media applications. It leverages state-of-the-art specialized libraries such as OpenCV, FFmpeg, ZeroMQ, picamera2, and yt_dlp, offering a powerful Multi-Threaded and Asyncio API framework. vidgear is designed to simplify video processing tasks, enabling developers to read, write, process, send, and receive video files, frames, and streams from various devices in real-time, often faster than underlying libraries. Its architecture is built around a concept of "Gears," each offering unique functionalities for different video processing needs.

Installation

Getting started with vidgear is straightforward. For detailed instructions on how to install the framework and its dependencies, please refer to the official documentation:

Installation Guide

Examples

vidgear offers a variety of "Gears" for different functionalities. Here are a couple of examples showcasing its powerful capabilities:

FFGear: AI Inference Optimization with Keyframes

FFGear is a multi-threaded, high-performance wrapper around DeFFcode's FFdecoder API, enabling real-time, low-overhead video frame decoding. It supports hardware-accelerated decoding, flexible pixel formats, and per-frame metadata extraction. A notable feature is its ability to optimize AI inference by processing only keyframes (I-frames), significantly reducing computational load.

# import required libraries
from vidgear.gears import FFGear
from ultralytics import YOLO

# Initialize YOLOv10-Nano model
model = YOLO("yolov10n.pt")

# Configure FFGear with per-frame metadata extraction enable
options = {"-extract_metadata": True}
stream = FFGear(
    source="test.mp4", frame_format="bgr24", logging=True, **options
).start()

# loop over
while True:

    # read data from stream
    output = stream.read()

    # check if end of stream
    if output is None:
        break

    # Unpack the frame and its associated metadata
    frame, meta = output

    # --- OPTIMIZATION STEP ---
    # We skip all non-keyframes to save processing power.
    # This ensures the model only runs on the most information-dense frames.
    if not meta.get("is_keyframe"):
        continue  # <-- Skips Non-key frames (P, B-frames)

    # Log keyframe details
    print(f"Keyframe #{meta['frame_num']} at {meta['pts_time']:.3f}s")

    # Perform AI Inference on keyframes (I-frames) only
    # Because we skip non-keyframes, this heavy task runs significantly less often.
    results = model(frame)

    # Annotate the frame with detection boxes and labels
    annotated_frame = results[0].plot()

    # {Insert your custom logic here, e.g., displaying/saving frames or triggering an alert}

# safely close video stream
stream.stop()

VideoGear: Real-time Video Stabilization

VideoGear provides a special internal wrapper around vidgear's exclusive Video Stabilizer class. It serves as a unified video-capture API, offering seamless access to CamGear, PiGear, and FFGear, along with their respective parameters. This makes it particularly useful for easily stabilizing both real-time and non-real-time video streams with minimal code.

# import required libraries
from vidgear.gears import VideoGear
import numpy as np
import cv2

# open any valid video stream with stabilization enabled(`stabilize = True`)
stream_stab = VideoGear(source="test.mp4", stabilize=True).start()

# open same stream without stabilization for comparison
stream_org = VideoGear(source="test.mp4").start()

# loop over
while True:

    # read stabilized frames
    frame_stab = stream_stab.read()

    # check for stabilized frame if Nonetype
    if frame_stab is None:
        break

    # read un-stabilized frame
    frame_org = stream_org.read()

    # concatenate both frames
    output_frame = np.concatenate((frame_org, frame_stab), axis=1)

    # put text over concatenated frame
    cv2.putText(
        output_frame,
        "Before",
        (10, output_frame.shape[0] - 10),
        cv2.FONT_HERSHEY_SIMPLEX,
        0.6,
        (0, 255, 0),
        2,
    )
    cv2.putText(
        output_frame,
        "After",
        (output_frame.shape[1] // 2 + 10, output_frame.shape[0] - 10),
        cv2.FONT_HERSHEY_SIMPLEX,
        0.6,
        (0, 255, 0),
        2,
    )

    # Show output window
    cv2.imshow("Stabilized Frame", output_frame)

    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

# close output window
cv2.destroyAllWindows()

# safely close both video streams
stream_org.stop()
stream_stab.stop()

Why use vidgear?

vidgear's motto is "Write Less and Accomplish More." It is designed to be a cross-platform, high-performance framework that provides a one-stop video processing solution for building complex real-time media applications in Python. It excels at reading, writing, processing, sending, and receiving video files, frames, and streams from various devices in real-time, often outperforming underlying libraries. Whether you are new to Python programming or an experienced developer, vidgear allows you to easily integrate and perform complex video processing tasks in your applications without extensive documentation, often with just a few lines of code.