Hello 👋!

I am Yusheng Dai, a PhD candidate at Monash University in Australia, working with Prof. Jianfei Cai (IEEE Fellow) and Prof. Qiuhong Ke. Before that, I completed my Master’s program at University of Science and Technology of China (USTC), working with Prof. Jun Du and Prof. Chin-hui Lee (IEEE Fellow).

My research focuses on Audio-Visual Foundation World Models, working toward an interactive metaverse driven by real-time video and sound generation. Representative works include:

• Omni2Sound (CVPR 2026 Highlight): A unified video-text-to-audio foundation model that achieves state-of-the-art on V2A, T2A, and VT2A with a simple DiT architecture.
• εar-VAE2 (arXiv Technical Report): A frequency-aware complex-spectrogram VAE for high-fidelity music reconstruction that compresses 48 kHz stereo music into 25 Hz latents and serves as the audio autoencoder of Qwen’s music generation model.
• CineDub (ACM MM 2026): The first scalable end-to-end cinematic dubbing model that operates on holistic video without preprocessing pipelines, jointly modeling speech, sound effects, and music across multi-speaker, off-screen, and combined generation scenarios.
• ControlAudio (ACL 2026 Oral): A controllable multi-event audio foundation model that produces millisecond-level temporally aligned audio from natural language descriptions.
• SaFa (ICCV 2025): Seamless long-form audio and panorama generation via latent swap joint diffusion, up to 20x faster than training-based methods.

Note: I am looking for collaborators to do great work in audio and speech — generation or understanding, frontend or backend. I bring strong research insights and sharp storytelling. If your work is potential enough to achieve high impact, email me. I am fast enough: I have gone to help my collaborators from zero context to a finished paper in one week, multiple times.

News

• [Aug. 2026]: Our work on scaling end-to-end multi-speaker video dubbing with coherent audio CineDub has been accepted by ACM MM 2026 (arXiv). Code, checkpoints, and benchmark are released: code, weights, benchmark.
• [Aug. 2026]: Our work on high-fidelity music reconstruction εar-VAE2 is released (arXiv, code): a complex-spectrogram VAE that compresses 48 kHz stereo music into 25 Hz latents and serves as the audio autoencoder of Qwen’s music generation model.
• [Apr. 2026]: Our work on unified video-text-to-audio generation Omni2Sound has been accepted by CVPR 2026 as a Highlight (top 15%).

Selected Publications [Google Scholar]