Liangcai Li
Spark CuDF Engineer at NVIDIA Semiconductor Technology Shanghai Co Ltd
Shanghai, Shanghai, China
Summary
🤩
Rockstar🎓
Top SchoolLiangcai Li is a Spark/cuDF engineer with nine years of experience building high-performance data processing at NVIDIA in Shanghai, specializing in GPU-accelerated Apache Spark and cuDF internals. He contributes to prominent open-source projects like NVIDIA/spark-rapids and rapidsai/cudf, improving CI/CD, JVM-Python data transfer performance, Pandas UDF correctness, and adding Java APIs and UTF8/string support. His work spans back-end development, native C++ integrations, and DevOps, reflecting a rare full-stack systems view across runtime, bindings, and testing. Prior experience at Marvell and a master’s in Electrical/Electronics from Huazhong University underpin his deep systems and hardware-aware software expertise. Colleagues rely on him for reliably shipping performance-sensitive features and for tackling cross-language interoperability challenges that aren’t obvious from commit messages alone.
9 years of coding experience
3 years of employment as a software developer
Master's degree, Electrical, Electronics and Communications Engineering, Master's degree, Electrical, Electronics and Communications Engineering at Huazhong University of Science and Technology
Stackoverflow
11reputation
6kreached
1answer
0questions
Github Skills (24)
apache-spark10
c-language10
python10
dataframes10
cudf10
java10
dataframe10
javas10
jna10
cuda10
cprogramming-language10
gpu-acceleration10
unit-testing9
cicd9
arrow-keys9
Programming languages (5)
Github contributions (5)
NVIDIA/spark-rapids
Jun 2020 - Jan 2023
Spark RAPIDS plugin - accelerate Apache Spark with GPUs
Role in this project:
Back-end Developer & DevOps Engineer
Back-end Developer & DevOps EngineerContributions:1163 reviews, 167 commits, 362 PRs in 2 years 7 months
Contributions summary:Liangcai primarily focused on enhancing the functionality of the `spark-rapids` project, which accelerates Apache Spark with GPUs. Their contributions involved modifications to the CI/CD pipeline, as evidenced by updates to deployment scripts, release processes, and Jenkins configuration, while also contributing to the core functionality. The user also worked on improving the performance by accelerating the data transfer between the JVM and Python for different plan execution. They also enabled and fixed tests, ensuring code quality and maintainability, specifically related to Pandas UDFs.
gpu-accelerationgpuapachebig-dataspark
rapidsai/cudf
Jan 2021 - Nov 2022
cuDF - GPU DataFrame Library
Role in this project:Back-end Developer
Back-end DeveloperContributions:117 reviews, 16 commits, 26 PRs in 1 year 9 months
Contributions summary:Liangcai primarily focused on adding new features and enhancing the Java interface within the `cudf` repository. Their work included implementing a Java interface for the 'explode' API, adding unit tests for the 'collect' aggregation with windowing, and supporting the creation of list scalars. They also contributed to the support for UTF8 strings and refactored the code for making a column from a scalar. The user's contributions involve changes across multiple files, including Java code, native C++ code, and test files.
cudadataframe-librarydata-analysiscppcudf
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.