Anish Shrigondekar
Software Engineer at Databricks
San Francisco Bay Area United States
Summary
🤩
Rockstar🎓
Top SchoolAnish Shrigondekar is a software engineer based in the San Francisco Bay Area with four years of professional experience focused on streaming and core data platforms. Currently at Databricks, he works on Core Streaming and has contributed backend improvements to high-profile open-source projects like Delta Lake and Apache Spark, enhancing CDC handling, state store reliability, and RocksDB memory management. His background includes systems-level work on ESX kernel storage at VMware and scalable platform engineering at Splunk, giving him a strong foundation in low-latency, stateful systems. Notably, his open-source contributions address subtle streaming edge cases—such as no-op merge handling and empty microbatch generation—demonstrating attention to correctness and performance in production data pipelines. He holds an MS in Information Networking from Carnegie Mellon and a BEng in Information Technology.
4 years of coding experience
10 years of employment as a software developer
Bachelor of Engineering (BEng), Information Technology, Bachelor of Engineering (BEng), Information Technology at Pune Institute of Computer TechnologyMaster of Science (MS), Information Networking, Master of Science (MS), Information Networking at Carnegie Mellon University
Github Skills (16)
rocksdb10
javas10
big-data10
apache-spark10
spark10
delta-lake10
java10
streaming10
data-engineering10
scala10
testing9
data-structure9
data-structures9
algorithms8
analytics8
Programming languages (3)
Github contributions (5)
apache/spark
Apr 2022 - Jan 2023
Apache Spark - A unified analytics engine for large-scale data processing
Role in this project:
Back-end Developer
Back-end DeveloperContributions:1333 reviews, 5 commits, 99 PRs in 9 months
Contributions summary:Anish contributed significantly to the Apache Spark project, focusing on enhancements to the state store, particularly concerning the transformWithState operator. Their work involved implementing schema validations and improvements related to memory management within the RocksDB state store provider. Additionally, they addressed issues with timer functionalities and incorporated the support for reading change feed for list state and map types. These changes aimed to improve the reliability, performance, and feature set of the state store used in streaming applications.
analyticspythondata-processingsqlapache
delta-io/delta
Sep 2022 - Oct 2022
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
Role in this project:Back-end Developer
Back-end DeveloperContributions:5 commits, 2 PRs, 2 comments in 1 month
Contributions summary:Anish primarily contributes to the `delta-io/delta` project by refactoring and enhancing Delta source functionality. Their work includes adding a `shouldSkip` field to the `IndexedFile` within `DeltaSourceCDCSupport` to optimize processing during no-op merges, and also updating offsets within the latest offset for no data changes in CDF case. Furthermore, the user has added a new test case to validate empty dataframe generation for microbatch processing of non-data operations. This suggests a focus on improving the performance and stability of Delta Lake's streaming capabilities, particularly with CDC and merging features.
analyticsprestodbflinkbig-dataspark
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.