Scott Sandre
Senior Software Engineer, Delta Ecosystem at Databricks
San Francisco, California, United States
Summary
🤩
Rockstar🎓
Top SchoolScott Sandre is a Senior Software Engineer on Databricks' Delta Ecosystem team with five years of experience designing and shipping core Delta Lake features for large-scale data platforms. He specializes in back-end and database engineering, driving performance optimizations (e.g., 45x init time and 8x CPU improvements) and building cross-engine connectors like Delta Flink and the Delta Kernel. His work on S3 multi-cluster writes, ACID-preserving Spark-less writers, and Iceberg compatibility reflects deep expertise in distributed storage, metadata consistency, and production reliability. Scott also built internal tooling and test systems that slashed CI times 30x, and contributes to the prominent open-source delta-io/delta project focusing on change data feed, data file management, and VACUUM integration. Based in San Francisco and trained at the University of Waterloo, he pairs a methodical problem-solving style with a knack for turning ambiguous performance challenges into measurable system improvements.
5 years of coding experience
3 years of employment as a software developer
Bachelor of Software Engineering Honours Co-op, Bachelor of Software Engineering Honours Co-op at University of Waterloo
Github Skills (10)
data-storage10
javas10
delta-lake10
java10
hadoop9
testing9
database-design9
parquet9
data-structures8
data-structure8
Programming languages (5)
Github contributions (5)
delta-io/delta
Sep 2020 - Jan 2023
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
Role in this project:
Back-end Developer & Database Engineer
Back-end Developer & Database EngineerContributions:8 releases, 1404 reviews, 106 commits in 2 years 4 months
Contributions summary:Scott primarily contributed to the back-end functionality of the Delta Lake project, focusing on features related to the storage layer and data processing, specifically around data file management and change data feed (CDF) capabilities. Their work involved refactoring existing classes and adding new features for features around CDC and improving test coverage around the data processing, while ensuring that they integrated seamlessly with existing data read/write functionalities. They also updated existing code to increase efficiency, in addition to improvements for the quality of testing, particularly around integration with VACUUM operations.
analyticsprestodbflinkbig-dataspark
delta-io/connectors
Oct 2020 - Jan 2023
This library allows Scala and Java-based projects (including Apache Flink, Apache Hive, Apache Beam, and PrestoDB) to read from and write to Delta Lake.
Contributions:3 releases, 817 reviews, 112 commits in 2 years 3 months
lakedeltaprestodbbeamflink
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.