Todd Lipcon
Distinguished Software Engineer at Apache Software Foundation
San Francisco, California, United States
Summary
🤩
Rockstar🎓
Top SchoolTodd Lipcon is a Distinguished Software Engineer with 17+ years building and hardening large-scale distributed systems, currently leading infrastructure work at Google after a multi-year stint on Spanner. He is a long-time Apache committer and PMC chair with deep expertise across the Hadoop ecosystem—HDFS, HBase, Kudu, Thrift and related projects—where his contributions range from production-facing durability and performance fixes to storage and query engine improvements. A strong back-end and DevOps practitioner, Todd blends low-level systems work (checksums, edit logs, RPCs) with pragmatic security and deployment engineering such as SASL integration and cluster tooling. He founded and shepherded Apache Kudu from a side project into a widely adopted storage engine, showing rare full-cycle product-to-production experience for an IC. Based in San Francisco, he pairs research-grade attention to correctness with a history of optimizing CPU and IO hotspots across benchmarks and real-world deployments.
17 years of coding experience
20 years of employment as a software developer
Sc.B Computer Science, Sc.B Computer Science at Brown UniversityWayland High School
Thai
Github Skills (50)
lzo10
hadoop-mapreduce10
hbase10
filesystem10
c-language10
python10
kerberos10
back-end-development10
apache-drill10
testing10
compression-algorithm10
big-data10
user-authentication10
hdfs10
authentication10
Programming languages (15)
Github contributions (5)
twitter/hadoop-lzo
Oct 2009 - Nov 2011
Refactored version of code.google.com/hadoop-gpl-compression for hadoop 0.20
Role in this project:
Back-end Developer
Back-end DeveloperContributions:48 commits, 1 comment in 2 years 1 month
Contributions summary:Todd primarily contributed to the `hadoop-lzo` repository, which is a compression library for Hadoop. Their work involved refactoring code, specifically related to the LZO index and codec implementations. The user also worked on distributed indexing functionality, improving the performance and usability of LZO compression within a Hadoop environment. This includes addressing issues like memory leaks and integrating with the CDH3b3 release.
gplhadoopcompressionrefactored
apache/hadoop
Jan 2011 - Apr 2019
Apache Hadoop
Role in this project:Back-end Developer
Back-end DeveloperContributions:136 commits, 2 PRs, 3 pushes in 8 years 4 months
Contributions summary:The user, Todd Lipcon, primarily contributes to the core of the Apache Hadoop project, specifically focusing on enhancing and maintaining the core HDFS (Hadoop Distributed File System) component. Their work involves improvements to data transfer, including the implementation of more robust checksum validation and adjustments to client read and write performance. Furthermore, the user is responsible for improving the robustness of the file system's performance, for instance addressing issues with the edit log in situations where journal nodes have gaps or fail to operate.
apachebig-datasparkhadoopjava
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.