Tianlun Zhang
MTS 2, Software Engineer at eBay
Shanghai, Shanghai, China
Summary
🤩
Rockstar🎓
Top SchoolTianlun Zhang is a seasoned software engineer with 14 years of experience building and optimizing large-scale big data and streaming systems, currently at eBay in Shanghai. He has deep backend expertise across Spark, Iceberg, Beam and Kafka integrations, and has shipped performance and correctness fixes in flagship projects like Apache Spark and Apache Iceberg. As an Apache committer (Beam and formerly Gearpump) he blends production engineering with open-source stewardship, having authored Kafka connectors, benchmark extensions (HiBench) and runtime improvements for stream processing engines. His work often focuses on fault tolerance, eventual consistency and operational observability—adding metrics pipelines and checkpointing to real-time flows. Previously he led platform efforts for machine learning and Hadoop distributions at VIP.com and Intel, where he helped boost Hadoop performance and built Storm/Gearpump benchmarking tools. Colleagues know him for pragmatic, low-level fixes that measurably improve throughput and reliability in complex distributed systems.
14 years of coding experience
12 years of employment as a software developer
Bachelor of Science (BS), Computer Software Engineering, Bachelor of Science (BS), Computer Software Engineering at East China Normal University
Stackoverflow
2,995reputation
312kreached
15answers
57questions
Github Skills (54)
storm10
benchmark10
file-management10
spark10
data-collection10
dataframes10
big-data10
kafka10
benchmarking10
message-queue10
java10
dataframe10
scala10
javas10
data-streaming10
Programming languages (17)
Github contributions (5)
gearpump/gearpump
Sep 2014 - Dec 2019
Lightweight real-time big data streaming engine over Akka
Role in this project:
Back-end Developer
Back-end DeveloperContributions:1 release, 473 commits, 364 PRs in 5 years 3 months
Contributions summary:Tianlun's commits focus on integrating Apache Kafka into the Gearpump streaming engine. They primarily developed a KafkaSpout component, which connects Gearpump with Kafka and allows for the streaming of data from Kafka topics. The user also contributed to example applications, such as the KafkaWordCount, showcasing the integration's functionality. Additionally, they addressed issues related to message handling and checkpointing within the Kafka connector.
stream-processingbigdatastreaming-datastreamingbig-data
apache/beam
Mar 2016 - Jul 2022
Apache Beam is a unified programming model for Batch and Streaming data processing.
Role in this project:Back-end Developer
Back-end DeveloperContributions:95 commits, 66 PRs, 4 pushes in 6 years 4 months
Contributions summary:Tianlun's contributions primarily focused on enhancing the Apache Beam project's core functionalities. Their work involved merging branches and fixing post-merge issues in Java-based runner implementations, particularly concerning the Gearpump runner. These changes involved modifications to the `SimpleDoFnRunner`, `SplittableParDo`, and `SimplePushbackSideInputDoFnRunner` classes, indicating a focus on improving the processing and execution of user-defined functions and state management within the Beam framework. The user also updated the Gearpump API and corrected build issues.
golangpythonstreaming-databeambatch
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.