Cheng Li
Member Of Technical Staff at Black Forest Labs
Bellevue, Washington, United States
Summary
🤩
Rockstar🎓
Top SchoolCheng Li is a Member of Technical Staff at Black Forest Labs with a decade of experience building high-performance training and inference systems for large language and vision models. He led full-stack optimizations at Databricks Mosaic AI to deliver the DBRX foundation model and partnered with NVIDIA to bring FP8 training into production, achieving substantial speedups over BF16/FP16. At Microsoft DeepSpeed he improved latency, throughput, and usability for production services like GitHub Copilot and DALL·E2, integrating techniques such as FlashAttention and sub-byte quantization. Cheng is an active open-source contributor—authoring llm-analysis and contributing memory-profiling and activation-checkpointing features to LLM Foundry, Composer, and DeepSpeed—to make GPU resource planning and scaling more practical. He holds a PhD background from UIUC and combines academic rigor with hands-on systems engineering for multi-thousand-GPU training runs. An often-overlooked strength is his track record of resolving low-level numeric and communication issues (e.g., FP8/TransformerEngine) that unlock real-world production performance.
9 years of coding experience
9 years of employment as a software developer
Master of Science (MS) Computer Science and Engineering, Master of Science (MS) Computer Science and Engineering at University of MichiganPhD Candidate Computer Science, PhD Candidate Computer Science at University of Illinois Urbana-ChampaignBachelor's degree Electrical and Electronics Engineering, Bachelor's degree Electrical and Electronics Engineering at Shanghai Jiao Tong University
Github Skills (40)
pytorch10
docker10
performance-analytics10
performance-monitor10
python10
scripting10
machine-learning10
bash10
sdp10
mlp10
dockers10
performance-measurement10
llm10
deeplearning-ai10
script10
Programming languages (6)
Github contributions (5)
deepspeedai/DeepSpeed
Jan 2021 - Nov 2022
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Role in this project:
ML Engineer
ML EngineerContributions:56 reviews, 115 commits, 85 PRs in 1 year 10 months
Contributions summary:Cheng's commits primarily focus on enhancing and debugging the DeepSpeed Flops Profiler, a crucial tool for measuring and optimizing the performance of deep learning models within the DeepSpeed framework. They implemented various features related to flops and MACs calculation, backward pass timing, and improved aggregated profiling outputs. Furthermore, the user generalized the model input format for the profiler's `get_model_profile()` API, contributing to its broader usability.
billion-parametersfinetuningtrainingmixture-of-expertszero
mosaicml/llm-foundry
Nov 2023 - Aug 2024
LLM training code for Databricks foundation models
Role in this project:ML Engineer
ML EngineerContributions:7 reviews, 16 PRs, 16 pushes in 9 months
Contributions summary:Cheng primarily contributed to the implementation and configuration of activation checkpointing within the LLM Foundry project, specifically targeting the MPT model. Their work included adding functionality to support activation checkpointing, refactoring code, and improving the flexibility and configurability of the checkpointing process. This involved modifying core model components, adding tests, and fixing errors, as well as integrating memory snapshots and OOM observers.
deep-learningllmneural-networksnlppytorch
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.