The scientist teaching AI to make learning music more accessible
Aug 18, 2026, 11.18 AM IST
For most people learning an instrument, there is a moment of quiet defeat: the song they love exists as a recording, but not as sheet music they can actually play, or the only arrangement they can find is written for someone far more advanced. Jingwei Zhao has spent the better part of a decade building artificial intelligence systems designed to make that moment obsolete.Zhao, who completed his PhD at the National University of Singapore's Sound and Music Computing Lab, one of the world's leading centers for music technology, under the supervision of Professor Ye Wang, works at the intersection of machine learning and music. It is a field where progress is measured not just in benchmarks, but in whether the output is something a musician would actually want to play. His research spans music representation learning, generative music modeling, and music information retrieval, with a particular focus on one of the field's hardest problems: automatic music arrangement.
"Generating music from scratch gets a lot of attention, but arrangement is a different challenge," Zhao says. "You're not inventing something new. You're taking music that already exists and reshaping it for a different instrument, a different ensemble, a different skill level, while keeping what makes it recognisable. That's what musicians actually need day to day."
His work has appeared at the field's most competitive venues. His 2021 system AccoMontage, presented at the International Society for Music Information Retrieval (ISMIR) conference, introduced a novel approach to piano accompaniment arrangement with style transfer, and its open-source release has become a widely used reference in the field. He followed it with papers at IJCAI on representation learning for multi-track music, and at NeurIPS, one of the world's premier machine learning conferences, on multi-track accompaniment arrangement via style prior modelling. His Beat Transformer model, which tracks beats and downbeats from music audio, has been adopted in both research and creative tools.
Zhao's path to the field began in Shanghai, where he graduated with honours from Shanghai Jiao Tong University's elite Zhiyuan Honours Program. A trained accordionist and baritone who sang with the SJTU student choir, he arrived at machine learning research with something many of his peers lacked: a working musician's ear. "I evaluate my models the way I'd evaluate a fellow performer," he says. "The math can be perfect and the music can still be wrong. You have to be able to hear the difference."
That dual fluency has made him a sought-after figure in the music AI community. He has served as a task captain for MIREX, the field's international benchmark evaluation exchange, is the Publications Chair for ISMIR 2026, and reviews for NeurIPS, ICLR, ICML, and ACM Multimedia. He has also attracted the attention of the world's largest music technology companies: Yamaha Corporation brought him into its R&D division as a research engineer in Japan, putting his arrangement research into contact with one of the world's most storied instrument makers.
Now, Zhao is bringing his research from the lab to the real world. He recently joined Songscription, a US-based education technology startup whose mission is to "empower musicians worldwide to play, share, and learn the songs they love." The company has developed AI that automatically transcribes audio into sheet music and recently raised funding to expand its research team. Andrew Carlins, Songscription's Co-Founder and CEO, explains the hire: "Jingwei is one of the best arrangement researchers in the world, and his work is what will let us meet the demand of music learners around the world."
At Songscription, Zhao is applying his research to making any piece of music playable by music learners at any level. "If a beginner pianist loves a song, the technology should meet them where they are," Zhao says. "A simplified left hand, a manageable key, the melody intact. That's personal music education, and it's finally within reach."
His most recent paper, on learning music style for piano arrangement through cross-modal bootstrapping, was accepted to ISMIR 2026, a signal that his move into industry has sharpened, rather than slowed, his research output.
For Zhao, the throughline is access. "Music has always had gatekeepers. You needed a teacher, an arranger, an ear trained over years," he says. "AI won't replace any of those. But it can hand every learner a bridge between the music they love and the music they can play. That's worth building."
"Generating music from scratch gets a lot of attention, but arrangement is a different challenge," Zhao says. "You're not inventing something new. You're taking music that already exists and reshaping it for a different instrument, a different ensemble, a different skill level, while keeping what makes it recognisable. That's what musicians actually need day to day."
His work has appeared at the field's most competitive venues. His 2021 system AccoMontage, presented at the International Society for Music Information Retrieval (ISMIR) conference, introduced a novel approach to piano accompaniment arrangement with style transfer, and its open-source release has become a widely used reference in the field. He followed it with papers at IJCAI on representation learning for multi-track music, and at NeurIPS, one of the world's premier machine learning conferences, on multi-track accompaniment arrangement via style prior modelling. His Beat Transformer model, which tracks beats and downbeats from music audio, has been adopted in both research and creative tools.
Zhao's path to the field began in Shanghai, where he graduated with honours from Shanghai Jiao Tong University's elite Zhiyuan Honours Program. A trained accordionist and baritone who sang with the SJTU student choir, he arrived at machine learning research with something many of his peers lacked: a working musician's ear. "I evaluate my models the way I'd evaluate a fellow performer," he says. "The math can be perfect and the music can still be wrong. You have to be able to hear the difference."
That dual fluency has made him a sought-after figure in the music AI community. He has served as a task captain for MIREX, the field's international benchmark evaluation exchange, is the Publications Chair for ISMIR 2026, and reviews for NeurIPS, ICLR, ICML, and ACM Multimedia. He has also attracted the attention of the world's largest music technology companies: Yamaha Corporation brought him into its R&D division as a research engineer in Japan, putting his arrangement research into contact with one of the world's most storied instrument makers.
Now, Zhao is bringing his research from the lab to the real world. He recently joined Songscription, a US-based education technology startup whose mission is to "empower musicians worldwide to play, share, and learn the songs they love." The company has developed AI that automatically transcribes audio into sheet music and recently raised funding to expand its research team. Andrew Carlins, Songscription's Co-Founder and CEO, explains the hire: "Jingwei is one of the best arrangement researchers in the world, and his work is what will let us meet the demand of music learners around the world."
At Songscription, Zhao is applying his research to making any piece of music playable by music learners at any level. "If a beginner pianist loves a song, the technology should meet them where they are," Zhao says. "A simplified left hand, a manageable key, the melody intact. That's personal music education, and it's finally within reach."
His most recent paper, on learning music style for piano arrangement through cross-modal bootstrapping, was accepted to ISMIR 2026, a signal that his move into industry has sharpened, rather than slowed, his research output.
For Zhao, the throughline is access. "Music has always had gatekeepers. You needed a teacher, an arranger, an ear trained over years," he says. "AI won't replace any of those. But it can hand every learner a bridge between the music they love and the music they can play. That's worth building."