Ankur Dave
Staff Software Engineer at Databricks
Pittsburgh, Pennsylvania, United States
Summary
🤩
Rockstar🎓
Top SchoolAnkur Dave is a Staff Software Engineer at Databricks with 16 years of experience building high-performance distributed systems and query engines, currently focused on Photon for Apache Spark SQL. He earned a PhD in Computer Science from UC Berkeley's RISELab, where his research on large-scale analytics was guided by Ion Stoica and Raluca Ada Popa, and previously holds a BS in EECS from Berkeley. Ankur has deep backend and algorithms expertise demonstrated by significant open-source contributions to Apache Spark—improving GraphX APIs, implementing core graph operations, and adding disk-spilling caching to enhance reliability and performance. He blends research rigor with production engineering, shipping optimizations that avoid recomputation and strengthen core data-processing primitives. Based in Pittsburgh, he pairs academic depth with a track record of practical systems work across internships and research roles at leading tech organizations.
16 years of coding experience
12 years of employment as a software developer
Doctor of Philosophy (Ph.D.) Computer Science, Doctor of Philosophy (Ph.D.) Computer Science at University of California, BerkeleyHigh School, High School at Interlake High School
Stackoverflow
106reputation
6kreached
4answers
0questions
Github Skills (19)
caching10
algorithms10
spark10
data-serialization10
java10
serialization10
scala10
javas10
performance-optimization9
error-handling9
unit-test9
parallel-processing8
rdd6
spark-graphx6
bash6
Programming languages (7)
Github contributions (5)
mesos/spark
Apr 2011 - Sep 2013
Lightning-fast cluster computing in Java, Scala and Python.
Role in this project:
Back-end Developer
Back-end DeveloperContributions:42 commits in 2 years 5 months
Contributions summary:Ankur's primary contribution involves implementing a disk-spilling cache mechanism within the Spark framework, specifically the `DiskSpillingCache` class. This involved adding core functionality for spilling entries to disk when memory is exhausted, as well as incorporating error handling and addressing previously identified feature gaps. The user also refactored the `BoundedMemoryCache` class, and made various improvements to the Bagel framework. These changes aimed to enhance Spark's performance and reliability.
pythoncluster-computinglightningsparkscala
apache/spark
Sep 2014 - Apr 2022
Apache Spark - A unified analytics engine for large-scale data processing
Role in this project:Back-end Developer & Algorithms Engineer
Back-end Developer & Algorithms EngineerContributions:21 reviews, 10 commits, 16 PRs in 7 years 8 months
Contributions summary:Ankur's contributions centered on enhancing the Apache Spark GraphX library. They implemented an `aggregateNeighbors` method returning a Graph and updated the API of existing functions. The changes include the implementation of other core functionalities of graph operations, like `mask`, `innerJoinVertices`, and other optimization enhancements like avoiding recomputation by caching and correct implementation of edge partitions. The user's work also involved adding comprehensive unit tests and example implementations to ensure code quality and functionality.
analyticspythondata-processingsqlapache
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.