Performance & quality vs MediaPipe

How Longpipe compares with Google’s MediaPipe selfie segmenter, the default most web apps use for virtual backgrounds. All the code and raw results are in the Longpipe repo, so you can re-run everything on your own hardware.

Three things decide how fast and how good a background effect is:

You can also run MediaPipe’s model on Longpipe’s engine with model: 'mediapipe-selfie', which separates the engine from the model.

Performance

Background replacement on a webcam clip at 360p, on an Apple M4 MacBook Pro with Chrome 157, Longpipe 0.0.18. Frames per second for the full transcode (decode → effect → H.264 encode), median of interleaved runs. Higher is better.

Model and engineWebGPUWebGL
MediaPipe selfie (landscape), tasks-vision GPU delegate–123
MediaPipe selfie (landscape), Longpipe engine379127
Longpipe xs344182
Longpipe small332188
Longpipe medium250153
Longpipe large15897
Longpipe xl8359

MediaPipe’s tasks-vision library has no WebGPU delegate. Its GPU delegate runs on WebGL. The Longpipe rows are the SDK as shipped, including the optical-flow network on in-between frames.

Numbers vary with hardware and browser; compare the ordering, not the absolute values.

Quality

Matte accuracy on the P3M-10k validation sets: 1,000 portrait photos with hand-labelled alpha mattes, scored at full resolution.

ModelMask resolutionP3M-500-P MADP3M-500-NP MADP3M-500-P IoUP3M-500-NP IoU
Longpipe xs384×2400.02100.02330.9230.926
Longpipe small384×2240.01700.01550.9370.954
Longpipe medium512×3200.01060.01020.9660.972
Longpipe large640×4000.01110.01000.9630.972
Longpipe xl1280×7680.00710.00800.9770.979
MediaPipe selfie (square)256×2560.03040.02490.9230.940
MediaPipe selfie (landscape)256×1440.03220.02760.9170.934

Notes on fairness:

Run it yourself

Commands, methodology and the full results are in the benchmarks README and benchmarks/METHODOLOGY.md.