Arup De
Senior Software Engineer at LinkedIn
San Jose, California, United States
Summary
👤
Senior🎓
Top SchoolArup De is a Senior Software Engineer based in San Jose with a Ph.D. in Computer Science and over a decade of experience spanning research and production engineering in storage, distributed systems, and accelerated deep learning. He has driven system-level innovations at LinkedIn and Western Digital after research roles at Lawrence Livermore and UC San Diego, bridging academic rigor with production-grade design. His work focuses on accelerating large-scale distributed deep learning by leveraging emerging NVM technologies, NVMe/NVDIMM, GPGPUs, RISC-V, and FPGA prototyping to build scalable, efficient training systems. Arup blends low-level hardware insight with distributed system software expertise—evident in contributions that improved reliability and failure handling for TonY, enabling more robust Hadoop-based deep learning deployments. Known for pragmatic problem-solving, he frequently translates esoteric research into tangible platform improvements that reduce operational fragility and boost performance.
6 years of coding experience
12 years of employment as a software developer
University of California San DiegoIndian Institute of Technology KanpurMS, Computer Engineering, MS, Computer Engineering at University of California, Santa Barbara
Github Skills (10)
javas10
hadoop10
java10
deeplearning-ai9
machine-learning9
deep-learning9
tensorflow8
dockers7
docker7
api-design6
Programming languages (4)
Github contributions (5)
tony-framework/TonY
Feb 2020 - Nov 2020
TonY is a framework to natively run deep learning frameworks on Apache Hadoop.
Role in this project:
Back-end & DevOps Engineer
Back-end & DevOps EngineerContributions:3 releases, 2 reviews, 9 commits in 9 months
Contributions summary:Arup primarily focused on improving the application's reliability and deployment process. They fixed RPC port binding issues, ensuring the proper allocation and handling of network ports. The user also implemented a fix to handle worker failures more gracefully during training, allowing for the continuation of successful training runs. Additionally, the user addressed issues related to archive extracting and implemented a mechanism to avoid accumulation of container requests, improving overall stability. They bumped up the Hadoop version.
horovoddeep-learningmachine-learningapachehadoop-yarn
arde171/TonY
Feb 2020 - Feb 2020
TonY is a framework to natively run deep learning frameworks on Apache Hadoop.
Contributions:5 pushes, 1 branch in 6 days
deep-learningmachine-learningapachehadoopframeworks
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.