Sing a piece and see exactly where you went wrong, right on the sheet music.
Built in 24 hours at HackUMass XIV.
All of us are musicians, and we've all had the same problem: practicing alone, you don't always know what you're getting wrong. A note is a little flat, an entrance is a bit late, and nobody's there to catch it. We started with singing, where it's the hardest to hear yourself.
Upload your sheet music and record yourself singing it. Gensing listens to the recording and gives you the sheet music back with every note colored by how you did:
| Mark | Meaning |
|---|---|
| In tune | Right note, right pitch |
| Out of tune | Right note, but sharp or flat |
| Wrong note | You sang a different note |
| Early / late | Right note, off the beat |
| Skipped | You didn't sing it |
| Extra | You sang something that isn't written |
You also get an overall score (pitch, intonation, rhythm, completeness) and short tips on what to work on. Click any note to hear how you sang it.
Most singing apps only show a pitch line. Gensing tells you which note went wrong, how, and where in the piece.
- Free mode (default): sing at your own tempo; rhythm is judged against your own pulse.
- Strict mode: sing to the score's tempo after a one-bar count-in.
- Starting note: plays your first pitch so you start in the right key.
sheet music ─► notes you should sing ─┐
├─► match ─► grade ─► colored score + score + tips
recording ──► pitch ─► notes you sang ┘
- Read the score. music21 turns the sheet music into a list of expected notes.
- Hear the voice. librosa's pYIN tracker follows your pitch every 10 ms, and our own algorithm turns that curve into the notes you actually sang.
- Match them up. An edit-distance alignment (the same idea a spell checker uses) pairs sung notes with written ones, so one skipped or extra note doesn't throw off the rest. It also handles singing an octave up or down, recording delay, and notes cut in two.
- Grade. Each note is checked for pitch and timing, and the results roll up into your score.
- Coach. Claude reads the graded results and writes short tips. All the grading happens in our code; the AI only explains it. If Claude isn't reachable, rule-based tips take over.
The frontend draws the score with OpenSheetMusicDisplay and colors each notehead from the backend's results.
- Frontend: React, TypeScript, Vite, OpenSheetMusicDisplay, Web Audio API
- Backend: Python, FastAPI, music21, librosa, NumPy
- AI: Claude API
Needs Python 3.11+ and Node 20+. From the repo root:
npm run setup # once: installs everything
npm run dev # starts backend (:8000) and frontend (:5001) togetherOpen http://localhost:5001 and load samples/twinkle-twinkle/score.musicxml to try it.
For AI tips, copy backend/.env.example to backend/.env and set ANTHROPIC_API_KEY. Without it,
Gensing uses the rule-based tips.
For developers
npm testruns the backend tests;npm run check:takesruns every test recording through the pipeline and checks it against its answer key.npm run synthregenerates the test recordings: generated "sung" takes of each piece insamples/(perfect, with planted mistakes, an octave down, and delayed), each with an answer key.contracts/has a sample JSON for every hand-off between stages.- Design and decisions:
docs/DECISIONS.md(the spec),docs/PIPELINE_NOTES.md,docs/ALIGNMENT_EXPLAINED.md. - AI assistants: start with
AGENTS.md.
- Tune the algorithm to the style of music you're singing
- Support more complex classical and opera singing
- Expand to other instruments
Jason (frontend), Jonathan (audio + server), Mehul (music logic)
Sheet music rendering: OpenSheetMusicDisplay (BSD-3-Clause).
Piano samples for the starting note: Salamander Grand Piano by Alexander Holm, CC-BY 3.0. MP3 copies from Tone.js.
