Marvin Zhang
Sr Software Engineer at Baidu, Inc.
Chaoyang District, Beijing, China
Summary
🤩
Rockstar🎓
Top SchoolMarvin Zhang is a senior software engineer with 11 years of experience building large-scale distributed systems and storage solutions, currently contributing to Baidu Cloud's distributed storage. He brings deep hands-on expertise in Spark, HDFS, Cassandra, Kafka and Cosmos, alongside kernel and file system knowledge across Linux and Windows, and is fluent in C/C++, Python, Scala and JavaScript. At Microsoft he built data-pipeline and analytics jobs for Office/Bing/Ads, and his earlier storage work at EMC and Intel underpins a strong systems engineering pedigree. Marvin is also the author of “Windows Driver Development Internals” and a core back-end contributor to Crawlab, where he added Celery-based asynchronous execution and robust crawler management APIs—showing he pairs systems-level insight with practical open-source tooling. Based in Chaoyang, Beijing, he blends low-level kernel expertise with cloud-scale data engineering to solve storage and processing challenges end-to-end.
11 years of coding experience
8 years of employment as a software developer
Master, Electric engineering, Master, Electric engineering at Beijing Institute of Technology
Stackoverflow
1reputation
0reached
0answers
0questions
Github Skills (13)
celery10
back-end-development10
apidoc10
python10
flask10
crud10
flask-ask10
api-design9
mongodb-database9
mongodb9
api9
apim9
go6
Programming languages (12)
Github contributions (5)
crawlab-team/crawlab
Feb 2019 - Jan 2023
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
Role in this project:
Back-end Developer
Back-end DeveloperContributions:42 releases, 11 reviews, 1383 commits in 3 years 11 months
Contributions summary:Marvin's contributions primarily involved implementing and extending back-end functionalities within the Crawlab platform. They introduced Celery integration to enable asynchronous task execution, including adding CRUD (Create, Read, Update, Delete) base functionality to manage database interactions. The user also worked on defining and implementing API endpoints for spiders, including an endpoint to execute spiders using Celery tasks. These efforts suggest a focus on improving the platform's back-end features for managing and running crawlers.
pythonweb-crawlerspiderscrapyd-uidistributed-web
crawlab-team/crawlab-vcs
Dec 2020 - Oct 2022
Version Control System (VCS) for Crawlab
Contributions:5 releases, 49 commits, 54 pushes in 1 year 10 months
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.