Gengliang Wang
Apache Spark Committer PMC Member at The Apache Software Foundation
San Francisco, California, United States
Summary
🤩
Rockstar🎓
Top SchoolGengliang Wang is a seasoned back-end engineer and Apache Spark committer with 13 years of experience building large-scale data systems from San Francisco. Currently at Databricks and a PMC member of Apache Spark, he has contributed deep fixes and features across Spark SQL and Delta Lake—improving Parquet data source performance, partition pruning, transaction locking, and test robustness. His work spans both core engine optimizations and pragmatic UX improvements like Spark website enhancements, reflecting a rare blend of low-level systems skill and product-aware polish. Trained at Zhejiang University (MS/BS in Computer Science), he brings consistent open-source stewardship and a knack for turning complex distributed systems edge cases into reliable production behavior.
13 years of coding experience
6 years of employment as a software developer
Master’s Degree, Computer Science, Master’s Degree, Computer Science at Zhejiang University
Github Skills (28)
spark-sql10
spark10
delta-lake10
back-end-development10
data-engineering10
css10
big-data10
ui-design10
java10
javas10
sql10
html10
parquet10
datatypes9
analytics9
Programming languages (12)
Github contributions (5)
apache/spark
Sep 2017 - Jan 2023
Apache Spark - A unified analytics engine for large-scale data processing
Role in this project:
Back-end Developer
Back-end DeveloperContributions:2478 reviews, 155 commits, 1262 PRs in 5 years 4 months
Contributions summary:Gengliang primarily worked on enhancing the Apache Spark SQL engine's capabilities related to data sources, particularly focusing on the functionality and efficiency of Parquet data sources. Their contributions involved implementing and optimizing key features, such as partition pruning and enhanced error handling within the Parquet ecosystem. The user's commits also touch upon the core components of Spark SQL, including the handling of various data types and the development of new functions. Furthermore, the user worked on enhancing the web UI for better user experience.
analyticspythondata-processingsqlapache
delta-io/delta
Jan 2020 - Dec 2022
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
Role in this project:Back-end Developer
Back-end DeveloperContributions:43 reviews, 11 commits, 7 PRs in 2 years 10 months
Contributions summary:Gengliang primarily contributed to the Delta Lake project by addressing compilation errors, refactoring tests, and implementing new configurations related to locking during commits. They focused on modifying core components such as `OptimisticTransaction.scala`, `DeltaSQLConf.scala`, and test suites like `CheckConstraintsSuite`. They also worked on refactoring aspects of `DeltaTable.scala` and improving error messages.
analyticsprestodbflinkbig-dataspark
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.