Sing2Song:
An Accompaniment Generation System Based on Solo Singing
Choi Sen Ho
1
*
, Chap Issac Fung
1
*
, Huicheng Zhang
1
*
, Yulun Wu
1
*
,
Yueqiao Zhang
1
, Zhili Tan
1
, Qiuqiang Kong
2
, Lui Siu Hang
1
, Yaolong Ju
3
#
1 The Central Media Technology Institute, Huawei, China
,
2 The Chinese University of Hong Kong, Hong Kong SAR, China
, 3 Great Bay University, Dongguan, China
,
* Equal Contribution
, # Corresponding Author
Project Overview:
In this project, we propose an enhanced pipeline addressing three key challenges:
- Robust processing of vocal inputs with performance variations
- Generation of musically coherent long-form accompaniments
- Professional-grade audio rendering with style-aware mixing
Our framework acts as a mixing role of improvisational accompanist, composer, and audio engineer, integrating:
- A robust MIR system to track beats, pitches, and musical sections of solo singing
- A knowledge-enhanced MIDI generator that combines music theory rules with style-specific improvisation patterns
- An adaptive audio synthesis engine that emulates professional mixing decisions
This pipeline enables studio-quality musical productions from users' singing while preserving their unique expressive characteristics, representing a significant step toward personalized music generation with no input length limitation.
We demonstrate the effectiveness of our system through qualitative and quantitative evaluations, showing superior alignment, expressiveness, and stylistic consistency compared to state-of-the-art baselines.
