Ananya Kumar
Research Lead At TBD Labs at Meta
Palo Alto, California, United States
Summary
👤
Senior🎓
Top SchoolAnanya Kumar is a research lead at Meta TBD Labs with 11 years of experience advancing large-scale ML and safety for next-generation reasoning models. Previously she was a core contributor to reasoning efforts at OpenAI (o1, o3, GPT-5) and completed a PhD in statistical machine learning at Stanford under Percy Liang and Tengyu Ma. Her work spans both theory and practice—from calibration and evaluation tooling in the widely used HELM framework to generative model and safety research at DeepMind. Based in Palo Alto, she combines deep academic rigor with hands-on engineering across research and production settings. Colleagues know her for turning nuanced evaluation metrics into robust, reproducible infrastructure that improves model reliability. She’s driven by applying AGI research toward beneficial, well-governed outcomes.
11 years of coding experience
2 years of employment as a software developer
NUS High School of Mathematics & ScienceComputer Science, Computer Science at Carnegie Mellon UniversityPhD Machine Learning, PhD Machine Learning at Stanford University
English, Hindi
Github Skills (6)
machine-learning10
nlp10
calibration10
python10
evaluation10
metric10
Programming languages (4)
Github contributions (5)
stanford-crfm/helm
Jul 2022 - Nov 2022
Holistic Evaluation of Language Models (HELM), a framework to increase the transparency of language models (https://arxiv.org/abs/2211.09110). This framework is also used to evaluate text-to-image models in HEIM (https://arxiv.org/abs/2311.04287) and vision-language models in VHELM (https://arxiv.org/abs/2410.07112).
Role in this project:
ML Engineer
ML EngineerContributions:42 commits, 3 comments, 1 issue in 3 months
Contributions summary:Ananya contributed to the evaluation framework of language models (HELM) by adding and refining metrics, particularly focusing on calibration within the context of multiple-choice and classification problems. Their work involved incorporating calibration statistics, such as ECE (Expected Calibration Error), and adjusting the metrics to handle different evaluation splits and problem types. The user also made modifications to handle multiple-choice scenarios within the HELM framework.
nlparxivabsberthelm
p-lambda/gradual_domain_adaptation
Aug 2019 - Apr 2021
Contributions:58 commits, 49 pushes, 1 branch in 1 year 8 months
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.