Šimon Podhajský
Head Of AI at Waypoint AI
Prague, Prague, Czechia
Summary
🤩
Rockstar🎓
Top SchoolŠimon Podhajský is Head of AI with 13 years of experience building evaluation-driven AI and data platforms, currently shaping production-grade LLM evaluation strategy and agent metrics from his base in Prague. A former Yale cognitive science and neuroscience researcher, he blends rigorous experimental methods with hands-on data engineering, ML tooling, and full-stack pragmatism to turn subjective “vibes” about model behavior into actionable, automated metrics. He designed multi-level evaluation frameworks, introduced grounded-theory error analysis and LLM-as-judge pipelines, and has a track record of operationalizing RAG, synthetic regression suites, and retrieval testing. At Pure Storage he shipped an Autogen-based regex/JSON generator and saved substantial developer time and costs through automation; on GitHub he’s contributed test and QA improvements to a public LLM evaluation framework. Comfortable at startups and public companies alike, he also speaks, podcasts, and teaches—translating technical depth into reusable practices and clearer decision-making.
12 years of coding experience
11 years of employment as a software developer
N/A, Gymnázium (Grammar School), N/A, Gymnázium (Grammar School) at Gymnazium Nad AlejiBachelor of Science (BS), Cognitive Science, Bachelor of Science (BS), Cognitive Science at Yale UniversityInternational Baccalaureate, International Baccalaureate at Open Gate Boarding School
English, Czech, Slovak, German
Stackoverflow
305reputation
21kreached
4answers
4questions
Github Skills (11)
pytest10
python10
testing10
heroku6
airflow-scheduler6
bash6
awk6
airflow6
cloudmailin6
ruby-on-rails6
langchain6
Programming languages (16)
Github contributions (5)
confident-ai/deepeval
Mar 2024 - Apr 2024
The LLM Evaluation Framework
Role in this project:
QA Engineer / Test Automation Engineer
QA Engineer / Test Automation EngineerContributions:2 PRs, 1 issue in 21 days
Contributions summary:Šimon primarily focused on modifying and enhancing the testing infrastructure of the repository. They implemented features to override settings within the `pytest.ini` file, adjusted the test execution by focusing on marks, and made adjustments to test definitions. Additionally, the user addressed a minor typo, indicating a commitment to code quality and detail. These commits reflect an effort to refine the test process and configuration.
evaluation-frameworkevaluation-metricsllm-evaluationllm-evaluation-frameworkllm-evaluation-metrics
shippy/dotfiles
Oct 2015 - Jul 2021
My dotfiles
Contributions:240 commits, 1 PR, 134 pushes in 5 years 10 months
dotfilesvimzsh
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.