ONNX & other runtimes
The Longpipe SDK runs its networks as hand-written WebGPU/WebGL2 shaders. If you want the same trained
models somewhere else (Python, a native app, a server, mobile, or onnxruntime-web), use the official
ONNX exports. They’re MIT licensed, like everything else, and are exported directly from the training
checkpoints that produce the SDK’s weights, so they’re the same models the SDK ships.
If you’re in the browser and want a MediaStream in and out, use EffectsPipeline. If you
want frames in and an alpha matte out in the browser without the pipeline, see the model layer.
Where to get them
Two identical copies:
- Hugging Face: huggingface.co/Longpipe/longpipe, the easiest
route from Python (
hf_hub_download), with the full model card. - Longpipe CDN:
https://cdn.longpipe.dev/models/v/0.0.5/onnx/, served withAccess-Control-Allow-Origin: *, so browsers can fetch the files directly (e.g. foronnxruntime-web).
Both use the same layout:
onnx/<tier>/matting_<orientation>.onnx canvas RGB -> refined alpha (what the SDK renders)
onnx/<tier>/matting_base_<orientation>.onnx base-res RGB -> alpha (faster, softer edges)
onnx/<tier>/face_<orientation>.onnx canvas RGB -> 5 face-keypoint heatmaps
onnx/<tier>/flow_<orientation>.onnx previous + current frame -> backward optical flow
onnx/landmark_mesh.onnx 256x256 face crop -> 478 face-mesh points
onnx/manifest.json shapes, sha256, parity vs PyTorch
<tier> is xs, small, medium, large or xl (see presets), and <orientation>
is landscape or portrait. Every file has a fixed input shape.
Inputs and outputs
All inputs are RGB in [0, 1], NCHW float32, batch 1. No mean/std normalization.
| Graph | Input | Output |
|---|---|---|
matting | rgb [1,3,H,W] at canvas res | alpha [1,1,H,W] in [0,1] |
matting_base | rgb [1,3,h,w] at base res | alpha [1,1,h,w] in [0,1] |
face | rgb [1,3,H,W] at canvas res | heatmaps [1,5,h’,w’], probabilities: left eye, right eye, nose, left mouth corner, right mouth corner |
flow | frames [1,6,H,W]: previous frame (channels 0–2), current frame (3–5) | flow [1,2,h’,w’]: backward flow in base-res pixels |
landmark_mesh | rgb [1,3,256,256] face crop | landmarks [1,478,2], crop coordinates 0–1 |
Canvas sizes (landscape, H×W): xs 240×384, small 224×384, medium 320×512, large 400×640, xl 768×1280. Portrait files swap H and W. Resize your frame to the canvas size before inference.
Example (Python)
import numpy as np, onnxruntime as ort
from huggingface_hub import hf_hub_download
from PIL import Image
sess = ort.InferenceSession(hf_hub_download("Longpipe/longpipe", "onnx/large/matting_landscape.onnx"))
H, W = sess.get_inputs()[0].shape[2:] # 400, 640
frame = Image.open("me.jpg").convert("RGB")
x = np.asarray(frame.resize((W, H), Image.BILINEAR), np.float32).transpose(2, 0, 1)[None] / 255
alpha = sess.run(None, {"rgb": x})[0][0, 0] # [H, W] in [0, 1]
What the ONNX files don’t include
The SDK does more than run these graphs. Each ONNX file contains its own copy of the tier’s encoder,
whereas the SDK shares one encoder pass across matting, face and flow. And temporal stability isn’t
in any single graph: the SDK warps the previous alpha along the flow output and blends it with the
fresh prediction through a flow-gated stabilizer (see architecture). The
Hugging Face model card describes that step in more detail,
plus the exact face-crop recipe the landmark model expects.