Allison Wang
Apache Spark Committer at The Apache Software Foundation
Mountain View, California, United States
Summary
🤩
Rockstar🎓
Top SchoolAllison Wang is an Apache Spark committer and Staff Software Engineer based in Mountain View with five years of experience building scalable data systems and developer-facing features. At Databricks she has driven PySpark improvements around Python UDFs, the Python Data Source API, and observability for GenAI-enabled troubleshooting, and previously led SQL UDF, AQE, and query optimization work in core Spark. Her open-source contributions to the flagship apache/spark repo include enhancing the Spark SQL planner, AQE plan introspection, and robust handling of non-deterministic and table functions—work that impacts large-scale analytics used across the industry. Earlier roles at Robinhood and internships across top tech companies gave her production experience in real-time financial systems and data lakes. She combines deep systems-level expertise with a user-focused approach to tooling that makes complex distributed query behavior more observable and debuggable.
5 years of coding experience
7 years of employment as a software developer
Bachelor’s Degree, Computer Science, minor in Statistics, Bachelor’s Degree, Computer Science, minor in Statistics at Carnegie Mellon University
Japanese, Chinese, English
Github Skills (7)
query-optimization10
apache-spark10
sql10
scala10
javas8
java8
data-analysis7
Programming languages (5)
Github contributions (5)
apache/spark
Oct 2020 - Jan 2023
Apache Spark - A unified analytics engine for large-scale data processing
Role in this project:
Back-end Developer
Back-end DeveloperContributions:972 reviews, 26 commits, 197 PRs in 2 years 3 months
Contributions summary:Allison's contributions focused on enhancing the SQL query engine within the Apache Spark project. The user added a feature to show the initial plan in AQE plan tree string and improved the handling of SQL UDF. They also worked on implementing and testing SQL features, including fixes and optimizations, such as the resolution of SQL table functions and support for non-deterministic functions. Their work involved modifying and extending the codebase related to query planning, analysis, and execution within the Spark SQL module.
analyticspythondata-processingsqlapache
allisonwang-db/spark
Jul 2020 - Mar 2025
Apache Spark - A unified analytics engine for large-scale data processing
Contributions:607 pushes, 206 branches in 4 years 9 months
analyticsdata-processingspark-sqlapachebig-data
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.