Xu Wenyi
Senior Software Engineer (Data Platform Team)
Singapore, Singapore
Summary
👤
Senior🎓
Top SchoolXu Wenyi is a Senior Software Engineer specializing in data platforms and infrastructure with 11 years of experience building high-throughput, low-latency streaming systems. Based in Singapore, he has led realtime Spark structured streaming and Hudi initiatives, optimized Kafka-based pipelines, and tuned large-scale data workflows across companies like Shopee, Dataminr, Bank of America, and now Motional. He combines deep hands-on expertise in Spark, Kafka, and microservices with a practical focus on performance, cost efficiency, and observability. With an MS in Electrical Engineering and a background in building monitoring tools and REST APIs, he often bridges data engineering and platform reliability in production environments. Notably, he has a track record of contributing internal big-data components to better suit daily operational needs, reflecting a preference for pragmatic, production-first solutions.
11 years of coding experience
7 years of employment as a software developer
Master of Science in Engineering Electrical Engineering, Master of Science in Engineering Electrical Engineering at Arizona State UniversityBachelor of Science (B.S.) Electrical and Electronics Engineering, Bachelor of Science (B.S.) Electrical and Electronics Engineering at Huazhong University of Science and Technology
Chinese, English
Stackoverflow
21reputation
2kreached
2answers
1question
Github Skills (66)
log-generator10
mlflow9
ai9
dummy-data9
stress-testing9
apache-spark9
model-management9
traffic9
lifecycle9
machine-learning9
incremental-learning9
dvc9
hashtag9
spike9
python8
Programming languages (5)
Github contributions (5)
xuwenyihust/Sales-Data-Pipeline
May 2018 - Jun 2018
A data pipeline for sales data.
Contributions:20 PRs, 122 pushes, 8 branches in 7 days
pipelinesingerdata-ingestiondata-pipelinessales
xuwenyihust/Visor
Nov 2016 - Dec 2016
A Real-time Apache log monitor using Kafka & Spark Streaming, with fake log generator.
Contributions:1 release, 2 commits, 181 pushes in 10 days
sinkmonitorspark-streaming-kafkalog-generatorapache
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.