Conglong Li
Senior Research Scientist at Google DeepMind
Tokyo, Japan
Summary
🤩
Rockstar🎓
Top SchoolConglong Li is a Senior Research Scientist at Google DeepMind Japan with a decade of experience blending systems engineering and AI research, focusing on improving thinking, multilingual, and multimodal capabilities in Gemini models. He holds a CMU PhD and previously advanced large-scale training and efficiency features on Microsoft’s DeepSpeed—contributing kernel fixes, 1-bit Adam/LAMB optimizers, and curriculum learning for transformer training. His work habitually combines rigorous performance analysis with algorithm and policy design to eliminate inefficiencies across deep learning, similarity search, distributed caching, networks, and architecture. Notably, he has translated research into production impact before—e.g., learned early-termination for ANN search and caching strategies that demonstrated substantial cost savings. Based in Tokyo, he brings both deep academic rigor and practical open-source contributions to scaling state-of-the-art models.
10 years of coding experience
13 years of employment as a software developer
Master of Science - MS, Computer Science, Master of Science - MS, Computer Science at Rice UniversityDoctor of Philosophy - PhD, Computer Science, Doctor of Philosophy - PhD, Computer Science at Carnegie Mellon University
English, Chinese, Japanese
Github Skills (28)
algorithm10
pytorch10
distributed-training10
python10
optimizers10
machine-learning10
transformer-models10
deepspeed10
deep-learning10
trainings10
gpu10
optimisation10
cuda10
compression10
modeling10
Programming languages (1)
Github contributions (5)
deepspeedai/Megatron-DeepSpeed
Oct 2021 - Dec 2022
Ongoing research training transformer language models at scale, including: BERT & GPT-2
Role in this project:
ML Engineer
ML EngineerContributions:82 reviews, 52 commits, 101 PRs in 1 year 2 months
Contributions summary:Conglong contributed to the implementation of curriculum learning within the Megatron-Deepspeed framework, enhancing training capabilities for transformer language models. They modified the training script to integrate curriculum learning, enabling sequence length adjustments during training. Additionally, the user updated the model to handle curriculum learning and prevent truncation of evaluation data, affecting the core training and evaluation pipelines. The commits also include improvements to tensorboard logging of training metrics.
nlptransformersbertongoingscale
deepspeedai/DeepSpeedExamples
Aug 2020 - Dec 2022
Example models using DeepSpeed
Role in this project:ML Engineer
ML EngineerContributions:30 reviews, 21 commits, 50 PRs in 2 years 4 months
Contributions summary:Conglong primarily contributes to the development and maintenance of examples related to DeepSpeed, a deep learning optimization library. Their work includes integrating and configuring 1-bit Adam and LAMB optimizers, as well as adapting training scripts for different hardware and networking setups (MPI, Ethernet, Infiniband). They also make modifications to existing scripts for BERT and GPT-2 models, including curriculum learning implementations, demonstrating a focus on efficient large-scale model training.
deep-learningpytorchdeepspeed
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.