Inspiration
In lecture halls, comedy clubs and conference rooms, a person with a question usually has to wait for a handheld microphone to be passed to them. Everyone else waits too, and the moment is gone. (lowkey awkward) Microphone arrays hung from the ceiling or set on tables could pick up a single voice in a room without anyone holding anything. Whether an array can pull one person out of a crowd depends on where it sits, how many microphones it has and what the room does to the sound, and that is hard to know without building it. We wanted a way to try layouts before buying or installing hardware.
What it does
BeamStage is a browser-based simulator for planning invisible microphone arrays.
- You build a room from a preset (lecture hall, comedy club or large conference room, each with a seated audience) or draw your own outline, and pick a wall material.
- You place speakers and mic arrays, give each speaker an audio clip, and choose which speaker each array should focus on.
- Running the simulation produces what every mic would record, with all the speakers talking at once.
- The listen bar plays either the original mic audio or a beamformed track that isolates the focus speaker from the recorded mixture.
How we built it
- Frontend: Next.js 16 with React 19, Redux Toolkit, three.js for the 3D view and Tailwind CSS 4. Next API routes proxy requests to the backend.
- Simulation: a Flask backend uses pyroomacoustics (the image-source method) to model reflections for shoebox and custom polygon rooms, in 2D or 3D.
- Localization: GCC-PHAT time-difference-of-arrival estimates and a position solver.
- Beamforming: a frequency-domain LCMV beamformer works only on the recorded mic signals. It keeps full gain toward the focus speaker and places nulls toward the other speakers. A mask across the beams then removes much of what the nulls miss, and spectral subtraction trims the late reverb tail.
- Hosting: the backend runs on Render and the frontend talks to it through its
/apiroutes.
Challenges we ran into
- No hardware: the physical array was not available, so everything had to be validated in simulation first.
- Separating overlapping speakers: an array can only cancel one fewer speaker than it has mics, and a single mic cannot separate anyone. Our first beamformer was a plain delay-and-sum, which barely separated the speakers.
- Reverb and echo: in highly reflective rooms the beamformed audio still has echo, because the reflections arrive from every direction.
- A hidden nonlinearity: the soft limiter on the simulated mic audio breaks the linear mixing that beamforming relies on, so the beamformer works on the signals before the limiter.
- Speed and size: requests carry 30 seconds of audio per speaker, and the free hosting tier is slow. We found the backend repeating the same position estimate for every speaker and cut that work, which took about 70% of the request time in one test.
Accomplishments that we're proud of
- A complete loop from drawing a room to hearing one speaker isolated from a crowd, entirely in the browser.
- In a dry (anechoic) test room, the beam stayed correlated with its own speaker (0.5 to 0.67) while the other speakers dropped to about 0.01.
- Spectral subtraction cut the echo tail by roughly 10 dB in a very reverberant test room, without losing the speech.
- Custom room drawing, three presets with seated audiences, and scene import and export.
What we learned
- How beamforming works in practice: steering vectors, covariance estimation, null placement and the limit of one null per extra mic.
- Reverberation matters more than geometry. A beam that is clean in a dry room only partly helps in a reflective one.
- Simulations can hide assumptions, like a limiter that quietly breaks linearity, so we measured results (for example with coherence) instead of trusting our ears.
- Deployment limits such as request size, timeouts and slow free-tier CPUs shape how much audio you can move through an app.
What's next for BeamStage
- Test the beamformer on a real microphone array and recordings.
- Estimate speaker positions from the audio instead of using the positions placed in the editor, and track speakers who move.
- Stronger dereverberation so reflective rooms sound cleaner.
- Process audio in real time rather than as 30-second clips.
- Phone network to be used as the microphones
Built With
- flask
- next.js
- pyroomacoustics
- python
- typescript