Audio denoise
Real-time speech denoising — suppress background noise (keyboards, fans, traffic, room hum) from a microphone while keeping the speaker’s voice clean.
It runs as a separate pipeline from the video effect: an AudioWorklet on the audio render thread, in parallel with the GPU video work. The two share nothing but the output MediaStream, so denoise adds no load to the video pipeline.
Hear it
The same ~10-second clip — a noisy microphone recording, then cleaned by each model. Headphones recommended.
Noisy input
DFN3 — dfn (high)
DFN3 int8 — dfnint8 (mid)
RNNoise — rnnoise (low)
Enabling it
Pass audio: 'denoise'. The input stream’s microphone track is denoised in place; pipeline.stream carries the cleaned audio track.
const stream = await navigator.mediaDevices.getUserMedia({ video: true, audio: true })
const pipeline = new EffectsPipeline(stream, {
background: 'blur',
audio: 'denoise',
})
videoEl.srcObject = pipeline.stream // denoised audio + processed video
pipeline.ready resolves on the first video frame and does not wait on audio — the mic passes through until the denoiser has loaded (a beat after init), then switches over automatically.
Models
Three models cover the hardware range. The default, 'auto', runs a tiny weight-free probe at init: it times the real network on the user’s device without downloading the multi-MB weights, then picks the best fit and downloads only that one.
model | tier | what it is |
|---|---|---|
'dfn' | high | DeepFilterNet3, full f32 — best quality |
'dfnint8' | mid | DeepFilterNet3 with int8 GRUs — smaller download, faster on weak hardware |
'rnnoise' | low | classic RNNoise — tiny, and the fallback where wasm SIMD isn’t available |
The two DeepFilterNet models are Longpipe’s hand-written WebAssembly port of DeepFilterNet3; RNNoise is Xiph’s RNNoise.
Force a specific model (or tier shorthand) to skip the probe:
new EffectsPipeline(stream, { audio: { denoise: { model: 'dfn' } } }) // explicit model
new EffectsPipeline(stream, { audio: { denoise: { model: 'high' } } }) // tier: high | mid | low
Options
interface DenoiseOptions {
model?: 'auto' | 'high' | 'mid' | 'low' | 'rnnoise' | 'dfn' | 'dfnint8' // default 'auto'
postFilterBeta?: number // DFN post-filter — extra suppression of residual noise (default 0.03)
gruLeak?: number // DFN recurrent-state drift bound (default 0.995)
enabled?: boolean // start denoising vs. passthrough (default true)
}
postFilterBeta and gruLeak tune the DeepFilterNet models ('dfn' / 'dfnint8') only; RNNoise ignores them.
Runtime control
pipeline.setDenoise(false) // passthrough — stop denoising (cheap to re-enable)
pipeline.setDenoise(true) // resume
pipeline.setDenoise({ postFilterBeta: 0.05 }) // tune DFN params live
Inspect what’s running and how it performs:
const s = pipeline.getAudioStats()
// { model: 'dfn', p50Ms: 0.4, p95Ms: 0.6, latencyMs: 12.5, sampleRate: 48000, active: true }
p50Ms / p95Ms are per-hop compute time (the budget is one 2.67 ms audio render quantum); latencyMs is the buffering the denoiser adds.
Audio-only (without the video pipeline)
EffectsPipeline needs a video track. For an audio-only stream — a participant with their camera off, or a mic-settings dialog where you only have the audio track — use the denoiser directly from its own entry point. It brings none of the video pipeline with it: no worker, no GPU, just the AudioWorklet.
import { AudioDenoiser } from 'longpipe/audio'
const mic = await navigator.mediaDevices.getUserMedia({ audio: true })
const denoiser = new AudioDenoiser(mic.getAudioTracks()[0], {
model: 'auto', // same options as DenoiseOptions above
onError: (msg) => console.warn(msg),
})
peerConnection.addTrack(denoiser.outputTrack) // live immediately (passthrough until loaded)
const loaded = await denoiser.ready // true = denoising, false = stayed in passthrough
outputTrack is live as soon as the constructor returns: it carries the raw mic until the model has loaded, then switches to denoised audio. ready resolves to true once denoising is live, or false if the model couldn’t load (onError says why). In that case audio keeps flowing as passthrough, so it never rejects.
The rest of the surface is what EffectsPipeline uses internally:
denoiser.setEnabled(false) // bypass (cheap to re-enable)
denoiser.setConfig({ postFilterBeta: 0.05 }) // tune DFN params live
denoiser.getStats() // same shape as getAudioStats()
await denoiser.resume() // resume a context the browser started suspended
denoiser.destroy() // closes its AudioContext
Browsers’ autoplay rules can start an AudioContext suspended until the user interacts with the page, and a suspended context processes no audio. The denoiser tries to resume on construction. If you construct it before any user interaction, also call denoiser.resume() from a click handler, such as your join or unmute button. It does nothing if the context is already running. If the context is still suspended shortly after construction, the denoiser calls onSuspended and resumes by itself on the next tap, click or key press. With EffectsPipeline, call EffectsPipeline.unlockAudio() first thing in the click that starts the call (before any await), or pipeline.resumeAudio() later; while suspended, the output carries the original mic audio (onAudioTrackChanged) and onError reports audio-suspended (non-fatal).
Assets come from the same default CDN as the video weights. If you self-host, pass weightsBaseUrl.
Sample rate
The models run at 48 kHz. If the browser gives the AudioContext a different rate, the SDK resamples in and out automatically — you don’t need to do anything. getAudioStats().sampleRate reflects the actual context rate.
Notes
- Denoise needs an audio track in the input — request
audio: trueingetUserMedia. With no audio track it’s a no-op (a non-fatalonErrorwithtype: 'audio'). Any denoise failure (assets, worklet, wasm) falls back to the original audio, never silence. - The DeepFilterNet models need wasm SIMD (every current browser has it); on a rare device without it,
'auto'falls back to RNNoise. - This cleans your mic for outbound audio — in a call you send
pipeline.streamto the other participant, whose playback is what they hear. If you monitor your own denoised audio locally, route it low-latency and use headphones (playing your live mic back is a feedback loop, the same as any mic monitor).