Diffusion Studio Core: Browser-Based Video Compositing Engine

Introduction

Diffusion Studio Core is a powerful, browser-based video compositing engine written in TypeScript. It provides a robust framework for fast media composition, optimized for video, audio, and image workloads. Leveraging the WebCodecs API and Canvas2DContext, it taps directly into hardware-accelerated processing within the browser, enabling both interactive playback for editing and high-fidelity rendering for final output. Developers often use it to build non-linear editors or other timeline-based media applications.

Installation

To get started with Diffusion Studio Core, you can install it via npm:

npm install @diffusionstudio/core

Once installed, you can import and use it in your TypeScript project:

import *s core from "@diffusionstudio/core";

const composition = new core.Composition();

Examples

Diffusion Studio Core offers a rich set of features for manipulating media. Here are a few examples demonstrating its capabilities:

Concatenate two videos

const sources = await Promise.all([
  core.Source.from<core.VideoSource>('/intro.webm'),
  core.Source.from<core.VideoSource>('/outro.mp4'),
]);

const layer = await composition.add(
  new core.Layer({
    mode: 'SEQUENTIAL'
  })
);

await layer.add(
  new core.VideoClip(sources[0], {
    range: [2, 8],
  })
);

await layer.add(
  new core.VideoClip(sources[1], {
    range: [2, 12],
  })
);

Apply basic transitions

new core.VideoClip(/** source **/, {
  transition: {
    duration: 1,
    type: 'dissolve',
  }
})

Mask an image

const mask = new core.RectangleMask({
  width: 640,
  height: 1080,
  radius: 100,
});

new core.ImageClip(/** source **/, { mask });

Animate your clips with key frames

new core.TextClip({
  text: "Hello World",
  align: 'center',
  baseline: 'middle',
  position: 'center',
  animations: [
    {
      key: 'rotation',
      frames: [
        { time: 0, value: 0 },
        { time: 2, value: 720 },
      ],
    },
  ]
});

Add basic effects to visual clips

new core.RectangleClip({
  position: 'center',
  delay: 6,
  duration: 4,
  effects: [
    {
      type: 'blur',
      value: 10,
    },
    {
      type: 'hue-rotate',
      value: 90
    }
  ]
})

Why use Diffusion Studio Core

Diffusion Studio Core is ideal if you are building a timeline-based application, such as a Non-Linear Editor (NLE), that requires in-browser video rendering. It's perfect for composing multiple assets into video or audio outputs and offers a framework-agnostic, efficient video engine compatible with popular frameworks like Svelte, Vue, Solid, and Angular.