Liquan Pei
Tech Lead at Chroma
Los Altos, California, United States
Summary
🤩
Rockstar🎓
Top SchoolLiquan Pei is a tech lead and founding engineer with 12 years building distributed data systems and data-intensive applications for large-scale internet companies, currently working on multimodal intelligent creation at ByteDance and co-founding Chroma. He has deep hands-on experience with streaming and connector ecosystems—contributing key features and exactly-once delivery implementations to Kafka Connect, Confluent connectors (HDFS, Elasticsearch), Schema Registry, and high-profile projects like Apache Kafka and Spark. At Pinterest he led a Kafka Streams-based rebuild of ads real-time infrastructure that scaled 10x input QPS and materially improved revenue and reliability, and at Confluent he drove connector architectures used in production at scale. His work spans back-end systems, serialization (Avro/SerDe), and ML infrastructure (Word2Vec in Spark), reflecting both low-level protocol rigor and applied ML engineering. Based in Los Altos, he blends research-grade algorithmic thinking (from his dual MS in CS and Physics) with pragmatic product delivery across startups and large platforms.
12 years of coding experience
9 years of employment as a software developer
BS Physics, BS Physics at University of Science and Technology of ChinaMS Computer Science, MS Computer Science at University of Massachusetts Amherst
English, Chinese, Japanese
Github Skills (43)
apim10
avro10
kafka-connect10
confluent10
spark10
word2vec10
python10
back-end-development10
api-design10
amazon-elasticsearch10
configuration-management10
big-data10
kafka10
machine-learning10
pyspark10
Programming languages (9)
Github contributions (5)
chroma-core/chroma
Jul 2023 - Sep 2024
the AI-native open-source embedding database
Role in this project:
Back-end Developer
Back-end DeveloperContributions:98 reviews, 142 PRs, 166 pushes in 1 year 2 months
Contributions summary:Liquan contributed to the core functionality of the chroma database. The user implemented support for metadata filtering on boolean values, which included modifying the SQLite metadata segment and related test files. The user's changes involved adding support for boolean values in the database schema, modifying the where clause to handle boolean values, and updating the existing tests to incorporate the new functionality. The user also worked on the go coordinator with database as the system catalog, adding tenant and database, which involved modifying the go coordinator APIs test.
document-retrievalembeddingsllms
apache/kafka
Oct 2015 - Jul 2018
Mirror of Apache Kafka
Role in this project:Back-end Developer
Back-end DeveloperContributions:52 commits, 45 PRs, 235 comments in 2 years 9 months
Contributions summary:The user, Liquan Pei, primarily contributed to the Kafka Connect project by implementing and refining features related to configuration management, REST APIs, and task handling within the Kafka Connect framework. Their work included adding REST APIs for listing connector plugins and exposing connector configurations. They also addressed bugs, refactored existing components, and improved the codebase related to source task offset committing. This included updates to documentation, refactoring and bug fixes.
apache-kafkakafka-consumerapachescalakafka
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.