Aaron Davidson
Senior Software Engineer at Databricks
Berkeley, California, United States
Summary
🤩
Rockstar🎓
Top SchoolAaron Davidson is a Senior Software Engineer based in Berkeley with 14 years of experience building and hardening distributed systems and cloud infrastructure. As tech lead of a Cloud Team at Databricks, he focuses on backend reliability, fault recovery, and orchestration for large-scale analytics workloads. His open-source contributions to high-profile projects like Apache Spark, Alluxio, and MLflow show a consistent emphasis on core system robustness—fixing scheduler persistence, edit-log serialization, and tricky file-handling/heartbeat bugs. Comfortable across backend, DevOps, and test automation, he pairs pragmatic engineering with attention to internal tooling and developer experience. Notably, his career blends production leadership with early-career full-stack product work at Google and Palantir, giving him a rare perspective from UI to cluster persistence.
14 years of coding experience
BS, Electrical Engineering and Computer Science, BS, Electrical Engineering and Computer Science at University of California, Berkeley
Github Skills (23)
unit-testing10
apache-spark10
python10
back-end-development10
testing10
distributed-systems10
alluxio10
zookeeper10
java10
scala10
javas10
orchestrate10
orchestration10
json9
thrift9
Programming languages (9)
Github contributions (5)
mlflow/mlflow
Jun 2018 - Jun 2020
Open source platform for the machine learning lifecycle
Role in this project:
Back-end Developer & DevOps Engineer
Back-end Developer & DevOps EngineerContributions:9 releases, 6 reviews, 157 commits in 2 years
Contributions summary:Aaron made several contributions to the MLflow project, primarily focused on improving the core functionality and usability of the tracking server. Their work included implementing instructions for the ``mlflow server`` command, adding a changelog for the 0.2.0 and 0.2.1 releases, and correcting documentation for the user interface. The user also addressed issues related to the code by avoiding git on PATH for running server and fixing Spark UDF when number of columns >= 10.
pythonlifecyclemlmachine-learningincremental-learning
apache/spark
Jan 2015 - Mar 2024
Apache Spark - A unified analytics engine for large-scale data processing
Role in this project:Back-end Developer
Back-end DeveloperContributions:1 review, 6 PRs, 235 comments in 9 years 4 months
Contributions summary:Aaron primarily focused on fixing issues within the `spark_ec2` and `standalone` scheduler components of the Apache Spark project. Their commits addressed problems related to empty security groups in `spark_ec2`, and implemented fault recovery for the standalone scheduler, including both file-based and ZooKeeper-based persistence. In addition, the user modified the underlying data structures and messages related to the internal workings of the cluster's components.
analyticspythondata-processingsqlapache
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.