Jeffrey Payne
Member Of Technical Staff (Data Engineering Architect)
San Diego, California, United States
Summary
👤
Senior🎓
Top SchoolJeffrey Payne is a data engineering architect and technical lead with 10+ years building large-scale, production data systems and ML workflows on open-source cloud platforms. Based in San Diego, he pairs hands-on engineering (Java, SQL, streaming and batch pipelines) with agile leadership to deliver robust, observable data products at companies like Bombora and 42 Lines. He contributes to Apache Airflow—improving GCP/Dataproc integration and Dataflow robustness—bringing practical open-source experience that directly enhances pipeline operability. With coursework in statistics, linear algebra, and mathematical modeling, he blends solid quantitative instincts with production pragmatism to solve complex data orchestration challenges.
10 years of coding experience
14 years of employment as a software developer
Almost a BS, Computer Science, Almost a BS, Computer Science at University of Nevada, Reno
Github Skills (15)
dataproc10
apache-airflow10
workflow-engine10
orchestration10
dataflow-programming10
orchestra10
dataflow10
data-pipelines10
python10
data-pipeline10
data-engineering10
gcp10
automations9
automation9
testing8
Programming languages (9)
Github contributions (5)
apache/airflow
Jul 2018 - Sep 2020
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Role in this project:
Data Engineer
Data EngineerContributions:5 PRs, 18 comments in 2 years 2 months
Contributions summary:Jeffrey primarily contributes to the Apache Airflow project by enhancing its integration with Google Cloud Platform (GCP) services, particularly Dataproc. They added functionality to retrieve the Dataproc job ID, which is crucial for monitoring and linking jobs in the Google Cloud Console. The user also fixed issues related to Google Cloud Storage interactions within the Dataflow operator, enhancing the robustness of data pipeline orchestration. They further improved operator functionality, allowing for custom job error states in Dataproc operations, enhancing their flexibility.
monitorpythonschedulerapacheprogrammatically
bomboradata/bombora-incubator-airflow
Mar 2018 - Sep 2019
Apache Airflow (Incubating)
Contributions:48 pushes, 10 branches in 1 year 5 months
incubatingemrapachebig-dataspark
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.