Manish Shetty
Researcher at METR
Berkeley, California, United States
Summary
🤩
Rockstar🎓
Top SchoolManish Shetty is a researcher based in Berkeley who measures frontier AI capabilities and studies methods to quantify AI’s transformative impacts, drawing on eight years of industry and academic experience. His PhD work at UC Berkeley focused on building evaluations and environments to elicit software engineering capabilities from models, and he has contributed to DSPy by improving assertion, error-handling, and bootstrapping mechanisms for more robust LM-driven programs. Manish has held research roles at METR, Microsoft, and DeepMind, blending rigorous experimental design with applied ML systems experience. He combines hands-on engineering of eval suites with a demonstrated ability to analyze validation failures and improve model reliability—skills that bridge research insight and practical tooling for AI assessment.
8 years of coding experience
4 years of employment as a software developer
Doctor of Philosophy, Computer Science, Doctor of Philosophy, Computer Science at University of California, BerkeleyBachelor of Technology - BTech, Computer Science, Bachelor of Technology - BTech, Computer Science at PES University
Tamil, Kannada, Hindi, English
Stackoverflow
1reputation
0reached
0answers
0questions
Github Skills (8)
machine-learning10
error-handling10
backtracking10
dnspy10
python10
computer-engineering9
nlp9
git7
Programming languages (6)
Github contributions (5)
stanfordnlp/dspy
Sep 2023 - Jan 2024
DSPy: The framework for programming—not prompting—language models
Role in this project:
ML Engineer
ML EngineerContributions:2 reviews, 6 PRs, 19 pushes in 4 months
Contributions summary:Manish contributed to the `dspy` repository by adding and modifying assertion support within the context of a question-answering framework. Their work involved introducing and refining mechanisms for error handling and backtracking, specifically through the implementation of `Assert` and `Suggest` primitives. These changes aimed to improve the robustness and reliability of DSPy programs, allowing for more effective debugging and self-correction within the language model workflows. Furthermore, they focused on analyzing validation failures and optimizing the bootstrapping process.
nlpbertknowledgepredictlanguage-models
manishshettym/tart
May 2023 - Jul 2024
tart: tensor abstracted relations and topologies
Contributions:5 releases, 33 PRs, 40 pushes in 1 year 1 month
graphsrelationstensors
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.