Colin Mccabe
Senior Software Engineer at LanceDB
Sunnyvale, California, United States
Summary
🤩
Rockstar🎓
Top School
Top expert inDistributed Data Systems and Analytics Technology
Colin Mccabe is a Senior Software Engineer with 17 years of experience specializing in distributed systems, storage, Linux kernel development, and big data, now based in Sunnyvale. He has driven core infrastructure and HDFS/Kafka reliability at Confluent and Cloudera and recently joined LanceDB to continue building scalable data systems. Colin is a steady open-source contributor to flagship projects like Apache Hadoop and Kafka, where his work addressed performance, JNI safety, short-circuit file I/O, and crypto integration. His background spans kernel and device-driver work (Lab126, Nvidia, Cozybit) through startup storage architecture at Locust and Dreamhost, giving him rare end-to-end expertise from firmware to distributed storage. Colleagues rely on him for pragmatic fixes that improve production robustness, and he often surfaces subtle performance and correctness issues that others miss.
17 years of coding experience
20 years of employment as a software developer
BS Computer Science Computer Engineering, BS Computer Science Computer Engineering at Carnegie Mellon University
Stackoverflow
4,230reputation
573kreached
14answers
0questions
stdout
top-1%
go
top-5%
command-line-interface
top-1%
Github Skills (34)
unit-testing10
filesystem10
command-line-interface10
configuration-management10
kafka10
hdfs10
metadata10
hadoop10
file-operations10
java10
javas10
jna10
openssl10
fileio10
html10
Programming languages (11)
Github contributions (5)
apache/kafka
Dec 2016 - Jan 2023
Mirror of Apache Kafka
Role in this project:
Back-end Developer
Back-end DeveloperContributions:2037 reviews, 423 commits, 1111 PRs in 6 years 1 month
Contributions summary:Colin's contributions focused on improving the admin client in the Apache Kafka repository. They implemented features to better handle errors during request processing and account for changes to the metadata, like a newly registered controller. Their work included enhancing the handling of dynamic configurations and implementing code for handling the re-registration of brokers in KIP-631 mode. This involved updates to several admin and controller classes for better reliability and compatibility.
apache-kafkakafka-consumerapachescalakafka
apache/hadoop-common
Mar 2014 - Aug 2014
Mirror of Apache Hadoop common
Role in this project:Back-end Developer
Back-end DeveloperContributions:67 commits in 4 months
Contributions summary:Colin primarily contributed to bug fixes and enhancements within the Hadoop common repository. Their work involved addressing issues related to block readers, available space choosing policies, short-circuit caches, and handling of short-circuit replica invalidation in the DataNode. The contributions include code modifications for test cases and implementing performance improvements. Additionally, the user was involved in integrating OpenSSL for AES-CTR crypto codec.
apachebig-datanetwork-serverhadoopjava
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.