ONNX & other runtimes

The Longpipe SDK runs its networks as hand-written WebGPU/WebGL2 shaders. If you want the same trained models somewhere else (Python, a native app, a server, mobile, or onnxruntime-web), use the official ONNX exports. They’re MIT licensed, like everything else, and are exported directly from the training checkpoints that produce the SDK’s weights, so they’re the same models the SDK ships.

If you’re in the browser and want a MediaStream in and out, use EffectsPipeline. If you want frames in and an alpha matte out in the browser without the pipeline, see the model layer.

Where to get them

Two identical copies:

Both use the same layout:

onnx/<tier>/matting_<orientation>.onnx       canvas RGB -> refined alpha (what the SDK renders)
onnx/<tier>/matting_base_<orientation>.onnx  base-res RGB -> alpha (faster, softer edges)
onnx/<tier>/face_<orientation>.onnx          canvas RGB -> 5 face-keypoint heatmaps
onnx/<tier>/flow_<orientation>.onnx          previous + current frame -> backward optical flow
onnx/landmark_mesh.onnx                      256x256 face crop -> 478 face-mesh points
onnx/manifest.json                           shapes, sha256, parity vs PyTorch

<tier> is xs, small, medium, large or xl (see presets), and <orientation> is landscape or portrait. Every file has a fixed input shape.

Inputs and outputs

All inputs are RGB in [0, 1], NCHW float32, batch 1. No mean/std normalization.

GraphInputOutput
mattingrgb [1,3,H,W] at canvas resalpha [1,1,H,W] in [0,1]
matting_basergb [1,3,h,w] at base resalpha [1,1,h,w] in [0,1]
facergb [1,3,H,W] at canvas resheatmaps [1,5,h’,w’], probabilities: left eye, right eye, nose, left mouth corner, right mouth corner
flowframes [1,6,H,W]: previous frame (channels 0–2), current frame (3–5)flow [1,2,h’,w’]: backward flow in base-res pixels
landmark_meshrgb [1,3,256,256] face croplandmarks [1,478,2], crop coordinates 0–1

Canvas sizes (landscape, H×W): xs 240×384, small 224×384, medium 320×512, large 400×640, xl 768×1280. Portrait files swap H and W. Resize your frame to the canvas size before inference.

Example (Python)

import numpy as np, onnxruntime as ort
from huggingface_hub import hf_hub_download
from PIL import Image

sess = ort.InferenceSession(hf_hub_download("Longpipe/longpipe", "onnx/large/matting_landscape.onnx"))
H, W = sess.get_inputs()[0].shape[2:]                   # 400, 640

frame = Image.open("me.jpg").convert("RGB")
x = np.asarray(frame.resize((W, H), Image.BILINEAR), np.float32).transpose(2, 0, 1)[None] / 255
alpha = sess.run(None, {"rgb": x})[0][0, 0]            # [H, W] in [0, 1]

What the ONNX files don’t include

The SDK does more than run these graphs. Each ONNX file contains its own copy of the tier’s encoder, whereas the SDK shares one encoder pass across matting, face and flow. And temporal stability isn’t in any single graph: the SDK warps the previous alpha along the flow output and blends it with the fresh prediction through a flow-gated stabilizer (see architecture). The Hugging Face model card describes that step in more detail, plus the exact face-crop recipe the landmark model expects.