Nishith Agarwal
Director, AI ML & Data
San Francisco Bay Area United States
Summary
🤩
Rockstar🎓
Top SchoolNishith Agarwal is a Director of AI/ML & Data with 13 years of experience building and operating large-scale distributed systems and data platforms from the Bay Area. He led Uber’s Big Data Compute and Storage platform—overseeing 500PB, 450K daily jobs and a global stack spanning Hudi, Spark, Hive and Kubernetes—then moved to Lyra Health to apply ML and data engineering to mental healthcare. A co-author and PMC for Apache Hudi, he has contributed core features like rollback, query-performance improvements and schema evolution that power incremental ETL at scale. Nishith combines hands-on backend and data-engineering chops with product leadership, hiring and cross-functional team building, and has invested in early-stage Data/ML startups as an angel. He pairs academic training from Georgia Tech and executive education at Harvard Business School Online with a pragmatic focus on reliability, GDPR compliance and cost-optimized data architectures. Notably, his experience spans both deep OSS stewardship and operational ownership of some of the world’s largest analytics fleets.
13 years of coding experience
3 years of employment as a software developer
Master's degree Computer Science, Master's degree Computer Science at Georgia Institute of TechnologyHBX | Executive Education Strategy & Leadership, HBX | Executive Education Strategy & Leadership at Harvard Business School Online
Github Skills (12)
javas10
avro10
apache-hive10
java10
data-engineering10
hadoop9
hbase9
data-pipelines8
big-data8
parquet8
data-pipeline8
data-lake7
Programming languages (2)
Github contributions (5)
apache/hudi
May 2017 - Mar 2022
Upserts, Deletes And Incremental Processing on Big Data.
Role in this project:
Back-end Developer & Data Engineer
Back-end Developer & Data EngineerContributions:469 reviews, 112 commits, 307 PRs in 4 years 10 months
Contributions summary:Nishith primarily contributed to the Apache Hudi project by implementing new features and improving existing ones. Their work focused on enhancing the storage and processing capabilities of the platform, specifically improving query performance and adding rollback functionality. This included enhancements to core components and integrating the system with Hive and HBase. The user's contributions also involved working with log files and schema evolution.
stream-processingbigdatahudiincrementalapacheflink
n3nash/hoodie
May 2017 - Oct 2022
Spark Library for Hadoop Upserts And Incrementals
Contributions:7 PRs, 802 pushes, 168 branches in 5 years 5 months
emrspark-mlsparkapache-sparkhadoop
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.