Towards a Speech Version of ChatGPT

Over the past few months, the capabilities of large language models (LLM) like ChatGPT have sparked widespread discussion. They possess a general processing ability, and they can often accomplish the desired tasks with the right instructions. Speech, teeming with rich and hierarchical information, offers different needs for diverse tasks. How far are we from the speech version of ChatGPT? What are we still missing for the birth of the speech ChatGPT? In this talk, I will share a series of the latest research findings in speech LM that could potentially lead us to the speech version of ChatGPT and discuss current challenges and potential solutions.