Puyuan Peng
Research Scientist at Meta
New York, New York, United States
Summary
🤩
Rockstar🎓
Top SchoolPuyuan Peng is a research scientist focused on SpeechLLMs, currently at Meta’s Superintelligence Lab after completing a PhD at UT Austin and seven years of research experience. His work spans zero-shot TTS and speech editing (VoiceCraft, ACL2024) and audio-visual generative models (ECCV2024), with VoiceCraft notably trending #1 on GitHub and amassing 8.4k stars. He pioneered prompt-based techniques for large speech models (PromptingWhisper) and has driven state-of-the-art results in visually grounded speech tasks, zero-resource recognition, and data-efficient representations. Puyuan blends strong statistical and mathematical foundations (MS Statistics, BS Mathematics) with hands-on experience training SpeechLLM pretrained models and audio tokenizers, and he has a track record of translating cognitive-inspired insights into scalable ML systems. An underappreciated strength is his consistent ability to produce transferable, zero-shot capabilities that bridge research demos and real-world VR and multimodal applications.
7 years of coding experience
4 years of employment as a software developer
Bachelor of Science - BS, Mathematics, Bachelor of Science - BS, Mathematics at Beijing Normal UniversityDoctor of Philosophy - PhD, Computer Science, Doctor of Philosophy - PhD, Computer Science at The University of Texas at AustinMaster of Science - MS, Statistics, Master of Science - MS, Statistics at University of Chicago
Github Skills (84)
sequence-to-sequence10
representation-learning10
apc10
ctc10
facebook10
sequence10
algolia9
flashlight9
self-supervised-learning9
conformer9
nlp9
python9
pytorch9
machine-learning9
jekyll-theme9
Programming languages (7)
Github contributions (5)
jasonppy/jasonppy.github.io
Apr 2021 - Jan 2025
Contributions:118 pushes, 1 branch in 3 years 10 months
jasonppy/VoiceCraft
Mar 2024 - Mar 2025
Zero-Shot Speech Editing and Text-to-Speech in the Wild
Contributions:13 reviews, 23 PRs, 37 pushes in 1 year
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.