Tianyi Zhang
Part Of Technical Stuff at Thinking Machines Lab
Palo Alto, California, United States
Summary
🤩
Rockstar🎓
Top SchoolTianyi Zhang is a Stanford-trained AI researcher and engineer with nine years of experience building and evaluating large language models and code-generation systems. As a final-year PhD candidate advised by Tatsunori Hashimoto, he has authored 20+ papers with over 20,000 citations and helped create influential open-source projects like Stanford Alpaca and the HELM evaluation framework. His contributions span core model evaluation (perplexity sampling, BERTScore integration, human-metric pipelines) and practical tooling for text-generation assessment, reflecting a blend of rigorous research and production-oriented engineering. He has worked on code-assistant benchmarks and reranking methods at Meta and applied ML to customer-support automation at ASAPP, showing a knack for turning research insights into deployable systems. Based in Palo Alto and now at Thinking Machines Lab, Tianyi combines deep academic pedigree with hands-on open-source impact that often surfaces in behind-the-scenes evaluation infrastructure.
9 years of coding experience
4 years of employment as a software developer
College Scholar Program and Computer Science, College Scholar Program and Computer Science at Cornell UniversityDoctor of Philosophy - PhD Artificial Intelligence, Doctor of Philosophy - PhD Artificial Intelligence at Stanford University
Github Skills (17)
pytorch10
summarization10
python10
machine-learning-models10
evaluation10
machine-learning10
bert10
natural-language-processing10
nlp10
tokenize9
tokenizer9
data-analysis9
cosine-similarity9
api8
apim8
Programming languages (5)
Github contributions (5)
Tiiiger/bert_score
Apr 2019 - Oct 2021
BERT score for text generation
Role in this project:
ML Engineer
ML EngineerContributions:47 commits, 19 PRs, 66 pushes in 2 years 5 months
Contributions summary:Tianyi primarily contributed to the core functionality of the BERTScore project, a tool for evaluating text generation using BERT. Their commits included implementing Python functions related to BERT embeddings, cosine similarity calculations, and IDF weighting, which are central to the score calculation. Additionally, the user introduced a plotting function to visualize the similarity matrix between candidate and reference sentences, improving the tool's utility. They also made updates to the setup and CLI scripts, and added a demo notebook for the project.
nlpdistilberttransformerssquadbert
stanford-crfm/helm
Jun 2022 - Nov 2022
Holistic Evaluation of Language Models (HELM), a framework to increase the transparency of language models (https://arxiv.org/abs/2211.09110). This framework is also used to evaluate text-to-image models in HEIM (https://arxiv.org/abs/2311.04287) and vision-language models in VHELM (https://arxiv.org/abs/2410.07112).
Role in this project:ML Engineer
ML EngineerContributions:29 commits in 4 months
Contributions summary:Tianyi contributed to the development and evaluation of language models within the HELM framework. Their work included implementing random window sampling for perplexity calculations and integrating the BERTScore metric for summarization tasks. The user also focused on setting up and debugging the computation of human evaluation metrics, including faithfulness, relevance, and coherence. Furthermore, the user made updates to accommodate new APIs and address edge cases in the evaluation process.
nlparxivabsberthelm
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.