Liwei Yang
Senior Software Engineer, Data
Japan
Summary
👤
Senior🎓
Top SchoolLiwei Yang is a Senior Software Engineer, Data with 11 years of experience building enterprise-grade big data and ML platforms across Asia. He has led and scaled cross-functional data teams, established new data organizations, and delivered high-impact solutions—from lakehouse architectures and MLOps to realtime streaming pipelines processing multi-GB daily workloads. His work at companies like LINE TV and Vpon includes designing ML platforms that automated model deployment and dramatically improved CTRs, and implementing CI/CD and Airflow-driven ETL orchestration for 50+ pipelines. He contributes to prominent open-source projects such as google-cloud-python and aws-sdk-pandas, improving datastore client functionality and S3 test coverage, reflecting a strong focus on code quality and data handling. Based in Japan, Liwei blends hands-on engineering with product-minded leadership, consistently reducing lead times and operational costs through automation and process improvements. He also publishes technical insights on his Medium blog, sharing practical learnings from building production data systems.
11 years of coding experience
8 years of employment as a software developer
National Chengchi University
Chinese, English
Stackoverflow
451reputation
140kreached
16answers
0questions
Github Skills (22)
unit-testing10
python10
data-engineering10
testing10
google-cloud-datastore10
aws10
amazon-s310
aws-s310
s3-bucket10
pandas9
pytest8
documentation7
csv6
hiveql6
hive6
Programming languages (12)
Github contributions (5)
aws/aws-sdk-pandas
May 2020 - Nov 2020
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
Role in this project:
Data Engineer
Data EngineerContributions:27 commits, 18 PRs, 13 comments in 6 months
Contributions summary:Liwei primarily contributed to the testing and functionality of the `aws-sdk-pandas` repository, specifically focusing on S3 object operations. They implemented test cases for deleting objects, checking object existence, listing directories and files, and describing objects. Furthermore, the user refactored naming conventions and added tests to cover more functionalities. These contributions suggest a focus on data handling and testing related to AWS S3 interactions within the context of a data engineering project.
amazon-sagemaker-notebookcsvpythonsqlserverpolars
googleapis/google-cloud-python
Nov 2014 - Aug 2017
Google Cloud Client Library for Python
Role in this project:Back-end Developer & QA Engineer
Back-end Developer & QA EngineerContributions:6 commits, 2 PRs, 1 comment in 2 years 8 months
Contributions summary:Liwei primarily contributed to the `google-cloud-python` repository by implementing and testing features related to the Google Cloud Datastore client library. Their work involved adding functionalities for retrieving entities, including handling specific edge cases such as odd lists and NoneType inputs. Furthermore, the user addressed coding style issues and corrected typos in documentation, thereby improving code quality, readability, and overall usability of the library.
gcpappenginepythongoogleclient-library
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.