Thomas Wang
Research Engineer at MistralAI
Summary
🤩
RockstarThomas Wang is a research engineer based in France with 9 years of experience specializing in large language model tooling, data pipelines, and performance-focused model engineering. He is an active open-source contributor to flagship projects such as Hugging Face Transformers and EleutherAI’s evaluation harness, where he has fixed model initialization bugs, added architectural support, and improved few-shot evaluation tasks. His work on BigScience and Megatron-DeepSpeed shows deep involvement in scalable data preprocessing, dataset merging, and sampling strategies for training huge transformer models. Thomas blends ML engineering with backend pragmatism—optimizing data loading, multiprocessing, and caching to accelerate research workflows. He also contributes less obvious but impactful fixes like shared-tensor support and FX baddbmm for improved runtime efficiency.
9 years of coding experience
Github Skills (30)
pytorch10
python10
architecture10
data-engineering10
data-set10
machine-learning10
datasets10
huggingface-transformers10
deep-learning10
tensorflow10
trainings10
natural-language-processing10
model-optimization10
data-processing10
transformer10
Programming languages (8)
Github contributions (5)
bigscience-workshop/Megatron-DeepSpeed
Jul 2021 - Jul 2022
Ongoing research training transformer language models at scale, including: BERT & GPT-2
Role in this project:
Back-end Developer
Back-end DeveloperContributions:339 reviews, 284 commits, 72 PRs in 11 months
Contributions summary:Thomas primarily contributed to refactoring and improving the preprocessing pipeline, splitting and merging data handling, and enhancing the codebase for better efficiency and stability. They separated merge scripts from the main processing script, reduced parameters, and streamlined data merging processes. The user also made improvements to the data loading process for faster preprocessing, and fixed potential issues during merging and with attention masks.
nlptransformersbertongoingscale
bigscience-workshop/bigscience
Aug 2021 - Jul 2022
Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.
Role in this project:ML Engineer & Data Engineer
ML Engineer & Data EngineerContributions:26 reviews, 196 commits, 34 PRs in 11 months
Contributions summary:Thomas contributed to the development and improvement of the BigScience project's model training and evaluation pipelines. They focused on generating dataset probability sampling configurations, modifying and fixing dataset probability generation scripts, and merging datasets based on language. Their work also included adapting evaluation scripts and integrating them with the project's workflow. This indicates a focus on data processing, model training, and result analysis within the context of large language models.
nlpplaceslurmmodelsmachine-learning
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.