Ziqi Liu
Senior Software Engineer at Databricks
Sunnyvale, California, United States
Summary
🤩
Rockstar🎓
Top SchoolZiqi Liu is a Senior Software Engineer at Databricks with nine years of experience building high-performance distributed systems and runtime optimizations, backed by a Master's from Carnegie Mellon and a Bachelor's from Tsinghua. He specializes in Spark Core and Databricks Runtime work, and has a track record of backend contributions to the flagship Apache Spark project—optimizing task/result handling and improving large-result efficiency using ChunkedByteBuffer. Prior roles at TuSimple focused on map engine and map infrastructure, where he delivered production-grade mapping systems for autonomous driving. Comfortable bridging research-quality algorithms (early ML/NLP internship work) and production engineering, he brings both deep systems thinking and practical performance tuning. Based in Sunnyvale, he combines rigorous academic training with hands-on open-source impact on one of the most widely used big-data engines.
9 years of coding experience
4 years of employment as a software developer
Bachelor, Information System&Management, 89.6/100, Bachelor, Information System&Management, 89.6/100 at Tsinghua UniversityMaster's degree, Information System, 3.73, Master's degree, Information System, 3.73 at Carnegie Mellon University
Github Skills (11)
javas10
big-data10
spark10
java10
scala10
sql9
performance-optimization9
algorithms8
algorithm8
data-structures8
data-structure8
Programming languages (10)
Github contributions (5)
apache/spark
Jul 2022 - Mar 2025
Apache Spark - A unified analytics engine for large-scale data processing
Role in this project:
Back-end Developer
Back-end DeveloperContributions:70 reviews, 25 PRs, 110 comments in 2 years 8 months
Contributions summary:Ziqi primarily contributed to the Apache Spark codebase by addressing core functionalities and performance. Their work involved optimizing the `take()` behavior by allowing customization of initial partition numbers. The user also implemented changes related to task result size calculations, excluding metadata. Furthermore, the user worked on improving the handling of large task results by replacing byte arrays with `ChunkedByteBuffer`. They also made changes to enhance the `TreeNode.generateTreeString`.
analyticspythondata-processingsqlapache
liuzqt/keyword_spotting
May 2017 - Dec 2017
Contributions:6 PRs, 374 pushes, 38 branches in 6 months
nlpchinesernndeep-learninglstm
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.