Fast, high-quality video effects in the browser — virtual backgrounds, face touch-up, auto-reframe, and mic noise removal in one open-source SDK.
Try the live demo → longpipe.dev/demo
Warning — This project is very new and still under active development. Expect API changes between versions and bugs.
npm install longpipe
import { EffectsPipeline } from 'longpipe'
EffectsPipeline.unlockAudio() // in your click handler, before any await: lets denoise start
const stream = await navigator.mediaDevices.getUserMedia({ video: true, audio: true })
const pipeline = new EffectsPipeline(stream, {
background: 'blur', // or an image, video, or solid color
touchup: true, // skin smoothing
reframe: true, // auto-frame the subject
audio: 'denoise', // mic noise removal
})
videoEl.srcObject = pipeline.stream // available immediately
await pipeline.ready // optional — resolves once the effect is liveNo bundler? See the single-file minimal example (CDN import, ~25 lines).
pipeline.stream is wired synchronously and emits the unprocessed input until the model is ready (~1–3 s on cold start), so users see live video the whole time. Model weights stream from cdn.longpipe.dev by default — no extra setup, and self-hostable.
- One model, many effects. Backgrounds, touch-up, and auto-reframe all run off a single shared encoder pass — enabling another effect doesn't cost another inference. Audio denoise runs in a parallel
AudioWorklet, off the GPU entirely. (Architecture) - More accurate than MediaPipe's selfie segmenter at every preset, and faster at
xs–large. Custom-trained models running as hand-written WebGPU/WebGL shaders in a zero-copy, fully-GPU pipeline. See the benchmarks. - Works everywhere. Chromium, Firefox, and Safari (desktop + iOS). WebGPU when available, WebGL2 fallback, per-browser frame transport handled internally.
- Adapts to the device. Five model presets (
xs–xl); autotune benchmarks the actual device at init and an adaptive controller swaps presets at runtime to hold 30 fps — on 10-year-old netbooks and the latest MacBook Pro alike. - Production pedigree. Built by the founder of Vectorly, a commercial effects SDK acquired in 2021 — a ground-up redesign for the WebGPU era.
Longpipe implements multiple downstream features (face effects, backgrounds) with one neural network, but if you want to compare it to another library, the closest would be the Mediapipe selfie model for background segmentation, from the @mediapipe/tasks-vision library.
Methodology, caveats (including where MediaPipe's model is faster, on LiteRT.js), the 720p, WebGL and IoU results, and scripts to reproduce everything: full analysis.
Full docs live at longpipe.dev/docs:
- Getting started
- API reference — all
EffectsPipelineoptions and methods - Architecture — the hydranet: one encoder, many heads
- Backgrounds · Touch-up · Auto-reframe · Audio denoise
- Presets & autotune
- Model layer — drive the model directly, without the pipeline
- Self-hosting weights
- ONNX & other runtimes
Machine-readable for LLMs and agents: llms.txt · llms-full.txt
- Background segmentation / virtual backgrounds
- Background noise removal (audio, separate pipeline)
- Face landmarks + touch-up
- Multi-face support
- Auto-reframe
- AR effects
- Lighting correction
SDK source code and Longpipe's pre-trained model weights are MIT-licensed — see LICENSE and WEIGHTS_LICENSE. You're free to self-host the weights inside other open-source or commercial projects under MIT-compatible terms. The optional MediaPipe selfie weights are Google's, under Apache-2.0 (NOTICE).
The weights were trained on permissively-licensed public datasets (P3M-10k, AISegment, COCO, OpenImages), Z-Image synthetics, and a custom webcam dataset collected with participants' explicit, informed consent. The full dataset list and provenance details: longpipe.dev/docs/licensing.