Tony Lee
PhD Candidate
Palo Alto, California, United States
Summary
🤩
RockstarTony Lee is a Stanford CS PhD candidate researching AI with a decade of software and ML engineering experience bridging research and production. He has held roles from senior SDE at Workday to R&D scientist at Stanford CRFM and a research engineer stint at Meta Superintelligence Labs, giving him deep expertise in model evaluation, API integration, and scalable systems. His open-source work includes backend contributions to HELM, a widely used framework for holistic language-model evaluation, where he integrated multiple vendor models and standardized tokenization/logprob metrics. He also improved robustness benchmarks in the WILDS project by tuning models and adding DistilBERT support, reflecting a practical focus on distributional shifts. Based in Palo Alto and advised by Percy Liang, Tony combines rigorous academic training with hands-on implementation skills across ML stacks. Colleagues value him for turning complex evaluation problems into maintainable, well-tested engineering solutions.
10 years of coding experience
Github Skills (13)
transformers10
apim10
hyperparameter-tuning10
pytorch10
distilbert10
machine-learning10
api10
trainings10
python10
modeling10
docker8
dockers8
huggingface7
Programming languages (3)
Github contributions (5)
stanford-crfm/helm
Dec 2021 - Mar 2023
Holistic Evaluation of Language Models (HELM), a framework to increase the transparency of language models (https://arxiv.org/abs/2211.09110). This framework is also used to evaluate text-to-image models in HEIM (https://arxiv.org/abs/2311.04287) and vision-language models in VHELM (https://arxiv.org/abs/2410.07112).
Role in this project:
Back-end Developer
Back-end DeveloperContributions:330 reviews, 1951 commits, 235 PRs in 1 year 2 months
Contributions summary:Tony's contributions focused on implementing features related to the integration of different models such as the Anthropic language model, the GooseAI language model, and Microsoft models. They modified client client code, added support for new models, including Luminous models, and the Google PaLM models, and handled the correct tokenization and logprob metrics from different model vendors. They added a separate test and did a lot of refactoring. The work demonstrates expertise in working with language model APIs and managing their differences.
nlparxivabsberthelm
p-lambda/wilds
Feb 2021 - Apr 2022
A machine learning benchmark of in-the-wild distribution shifts, with data loaders, evaluators, and default models.
Role in this project:ML Engineer
ML EngineerContributions:27 reviews, 426 commits, 29 PRs in 1 year 1 month
Contributions summary:Tony primarily focused on modifying and updating the machine learning models, specifically related to hyperparameter tuning and model selection. Their contributions included updating hyperparameters for datasets like iWildCam and Camelyon, and integrating support for DistilBERT within the project. They also fixed paths and imports related to the model implementations.
loadersshiftswildmachine-learningmachine-learning-benchmark
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.