Performance & quality vs MediaPipe
How Longpipe compares with Google’s MediaPipe selfie segmenter, the default most web apps use for virtual backgrounds. All the code and raw results are in the Longpipe repo, so you can re-run everything on your own hardware.
Three things decide how fast and how good a background effect is:
- Model: the neural network (MediaPipe selfie, or Longpipe’s presets xs → xl).
- Engine: what runs it in the browser. For MediaPipe that’s
@mediapipe/tasks-vision; Longpipe has its own hand-written WebGPU/WebGL shaders. - Pipeline: what happens around the network. Longpipe runs its full network on key frames and a small learned optical-flow network in between (see architecture).
You can also run MediaPipe’s model on Longpipe’s engine with model: 'mediapipe-selfie', which
separates the engine from the model.
Performance
Background replacement on a webcam clip at 360p, on an Apple M4 MacBook Pro with Chrome 157, Longpipe 0.0.18. Frames per second for the full transcode (decode → effect → H.264 encode), median of interleaved runs. Higher is better.
| Model and engine | WebGPU | WebGL |
|---|---|---|
| MediaPipe selfie (landscape), tasks-vision GPU delegate | – | 123 |
| MediaPipe selfie (landscape), Longpipe engine | 379 | 127 |
| Longpipe xs | 344 | 182 |
| Longpipe small | 332 | 188 |
| Longpipe medium | 250 | 153 |
| Longpipe large | 158 | 97 |
| Longpipe xl | 83 | 59 |
MediaPipe’s tasks-vision library has no WebGPU delegate. Its GPU delegate runs on WebGL. The Longpipe rows are the SDK as shipped, including the optical-flow network on in-between frames.
Numbers vary with hardware and browser; compare the ordering, not the absolute values.
Quality
Matte accuracy on the P3M-10k validation sets: 1,000 portrait photos with hand-labelled alpha mattes, scored at full resolution.
- MAD: mean absolute alpha error. Lower is better.
- IoU: overlap of the mask at 0.5. Higher is better.
| Model | Mask resolution | P3M-500-P MAD | P3M-500-NP MAD | P3M-500-P IoU | P3M-500-NP IoU |
|---|---|---|---|---|---|
| Longpipe xs | 384×240 | 0.0210 | 0.0233 | 0.923 | 0.926 |
| Longpipe small | 384×224 | 0.0170 | 0.0155 | 0.937 | 0.954 |
| Longpipe medium | 512×320 | 0.0106 | 0.0102 | 0.966 | 0.972 |
| Longpipe large | 640×400 | 0.0111 | 0.0100 | 0.963 | 0.972 |
| Longpipe xl | 1280×768 | 0.0071 | 0.0080 | 0.977 | 0.979 |
| MediaPipe selfie (square) | 256×256 | 0.0304 | 0.0249 | 0.923 | 0.940 |
| MediaPipe selfie (landscape) | 256×144 | 0.0322 | 0.0276 | 0.917 | 0.934 |
- Every Longpipe preset has a lower matte error than MediaPipe.
- On IoU, MediaPipe’s square model edges out xs on P3M-500-NP; small and up beat it.
Notes on fairness:
- MediaPipe’s model scores the same on either engine.
- MediaPipe selfie is a segmentation model, scored here on matting, because a virtual background shows the matte.
- Longpipe trained on P3M-10k’s face-blurred training split, so P3M-500-NP is the cleaner comparison.
- This is a single-image test, so it doesn’t measure flicker.
Run it yourself
- Speed:
benchmarks/performancein the repo, a browser harness (Node + Chrome) where every pipeline has the sameVideoFramein,VideoFrameout interface. - Quality:
benchmarks/quality/eval_p3m.py(Python, CPU), which runs Longpipe’s ONNX exports and Google’smediapipepackage on P3M-10k. You download the dataset yourself.
Commands, methodology and the full results are in the
benchmarks README and
benchmarks/METHODOLOGY.md.