Hongyu Ren
Research Scientist at PEKING UNIVERSITY
Summary
🤩
RockstarHongyu Ren is a Research Scientist with nine years of experience specializing in evaluation and benchmarking of machine learning models, currently contributing at OpenAI. He has hands-on expertise in building robust evaluation pipelines, notably extending the HELM framework to add a Wiki fact-completion scenario and automating data handling to improve reproducibility. His work on graph ML benchmarks (ogb) includes fixing evaluation code, enhancing knowledge-graph examples, and refining metrics and logging—demonstrating a pragmatic focus on trustworthy, auditable model assessment. Based in China, he blends research rigor with engineering pragmatism, favoring reproducible experiments and clear evaluation protocols that bridge academic frameworks and production needs.
9 years of coding experience
Github Skills (15)
graphdb10
machine-learning10
eval10
pytorch10
nlp10
deep-learning10
graph-neural-network10
graphml10
python10
evaluation10
natural-language-processing10
testing9
datasets9
graph-embedding9
knowledge-graph9
Programming languages (3)
Github contributions (5)
snap-stanford/ogb
Apr 2020 - Oct 2022
Benchmark datasets, data loaders, and evaluators for graph machine learning
Role in this project:
ML Engineer
ML EngineerContributions:24 commits, 35 pushes, 7 comments in 2 years 6 months
Contributions summary:Hongyu contributed to the graph machine learning benchmark repository by fixing bugs in the evaluation code, adding example code for a knowledge graph (KG) application, and updating logging information. The contributions include modifications to an evaluation metric and adding example code related to a specific dataset, demonstrating an involvement in the model development and evaluation processes. The user also updated examples for the wikikg dataset, suggesting the user worked with the model implementations and evaluating their performance on various datasets.
loadersdeep-learningmachine-learningbenchmarkgraph-neural-networks
stanford-crfm/helm
Feb 2022 - Mar 2022
Holistic Evaluation of Language Models (HELM), a framework to increase the transparency of language models (https://arxiv.org/abs/2211.09110). This framework is also used to evaluate text-to-image models in HEIM (https://arxiv.org/abs/2311.04287) and vision-language models in VHELM (https://arxiv.org/abs/2410.07112).
Role in this project:ML Engineer
ML EngineerContributions:17 commits in 17 days
Contributions summary:Hongyu contributed to the development and evaluation of language models within the HELM framework, specifically focusing on the Wiki scenario. They added a fact completion scenario, updated it to account for train/dev/test splits and different predicates, and integrated automatic data downloading. The user also modified the test and run specifications to incorporate the new Wiki scenario and adjust evaluation parameters, demonstrating a focus on model evaluation and testing.
nlparxivabsberthelm
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.