Teven Le Scao
Research Scientist at Mistral AI
Paris, Ile-de-France
Summary
🤩
Rockstar🎓
Top SchoolTeven Le Scao is a research scientist with nine years of experience building and scaling state-of-the-art NLP systems, currently working at Mistral AI after a multi-year research role at Hugging Face. He combines strong applied math training from École Polytechnique and an NLP master's from Carnegie Mellon with hands-on engineering—contributing bug fixes, tokenizer and pipeline improvements, and large-scale training tooling to flagship open-source projects like Hugging Face Transformers, Datasets, and the BigScience/Megatron-DeepSpeed stack. His work bridges research and production: from fixing subtle edge cases in µP and generation pipelines to designing sampling scripts and Slurm tooling for multilingual model training. Based in Paris, he also pursues creative “random art” experiments, reflecting a playful curiosity that often surfaces in pragmatic, well-documented code contributions.
9 years of coding experience
4 years of employment as a software developer
Master's degree, Applied Mathematics, Master's degree, Applied Mathematics at École PolytechniqueMaster's degree, Natural Language Processing, Master's degree, Natural Language Processing at Carnegie Mellon University
Github Skills (39)
transformers10
pytorch10
debug10
distributed-training10
language-model10
python10
evaluation10
model-driven10
machine-learning10
datasets10
transformer-models10
model-building10
eval10
deepspeed10
huggingface10
Programming languages (7)
Github contributions (5)
huggingface/datasets
Aug 2020 - Nov 2022
🤗 The largest hub of ready-to-use datasets for ML models with fast, easy-to-use and efficient data manipulation tools
Role in this project:
ML Engineer
ML EngineerContributions:5 reviews, 23 commits, 25 PRs in 2 years 3 months
Contributions summary:Teven contributed to the Hugging Face datasets repository by adding and modifying dataset loading scripts. They added new datasets such as HANS, a debiased subset of Winogrande, CS restaurants, Gutenberg time references, and glucose, demonstrating an understanding of dataset creation and integration. The user also updated existing scripts, including enriching the WebNLG dataset and handling specific XML formatting bugs, indicating an ability to work with diverse data formats and structures.
ml-modelstensorflownatural-language-processingmanipulationdata-science
bigscience-workshop/Megatron-DeepSpeed
Jul 2021 - Jun 2022
Ongoing research training transformer language models at scale, including: BERT & GPT-2
Role in this project:ML Engineer
ML EngineerContributions:31 reviews, 59 commits, 13 PRs in 10 months
Contributions summary:Teven contributed to the training and evaluation scripts for large language models, specifically focusing on floating-point operations (FLOPs) counting and model parameter analysis. They integrated DeepSpeed for distributed training and optimized scripts for single-node testing. Furthermore, the user worked on implementing features related to Hugging Face tokenizers and datasets, demonstrating proficiency in data preprocessing and model configuration within the Megatron-Deepspeed framework. The user's efforts included improving the overall training pipeline, especially around evaluation, tensorboard logging, and checkpointing.
nlptransformersbertongoingscale
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.