Sujen Shah
Platform Lead - NASA Multi-Mission Algorithm And Analysis Platform
Sierra Madre, California, United States
Summary
👤
Senior🎓
Top SchoolSujen Shah is a platform-focused software engineering leader with 11 years of experience and over 7 years at NASA JPL building large-scale, distributed science platforms used by NASA and ESA. As Platform Lead he manages a team delivering a cloud-native algorithm and analysis platform that processed 3+ million science jobs last year and supports high-impact uses like near-real-time fire monitoring and biomass estimates influencing international climate policy. He combines hands-on backend engineering (contributions to Apache Nutch and Sparkler) with infrastructure expertise—migrating deployments to AWS with Terraform and CI/CD—to keep mission-critical services performant and reliable. Sujen’s background spans ML systems, intelligent web crawling, and metalearning (DARPA D3M), reflecting a rare mix of research-grade tooling and production delivery. Based in Sierra Madre, CA, he is an active Apache PMC member and brings a proven track record of turning complex scientific workflows into scalable, operational platforms.
11 years of coding experience
8 years of employment as a software developer
Master of Science (MS) Computer Science, Master of Science (MS) Computer Science at University of Southern CaliforniaBachelor of Engineering (B.E.) Computer Engineering, Bachelor of Engineering (B.E.) Computer Engineering at D J Sanghvi College of Engineering
English, Hindi, Gujarati
Stackoverflow
270reputation
17kreached
12answers
1question
Github Skills (29)
api-rest10
search-engine10
apache-spark10
web-crawler10
webscraper10
api-design10
restful-api10
nutch10
information-retrieval10
solr10
webscraping10
webservice10
java10
scala10
crawling10
Programming languages (5)
Github contributions (5)
apache/nutch
Sep 2015 - Oct 2016
Apache Nutch is an extensible and scalable web crawler
Role in this project:
Back-end Developer
Back-end DeveloperContributions:13 commits, 25 PRs, 46 comments in 1 year
Contributions summary:Sujen Shah primarily worked on enhancing the REST API functionality for Apache Nutch. Their contributions involved implementing endpoints for reading sequence files, refactoring configuration endpoints, and addressing warnings related to the REST API. They also implemented new features for the seed management, including endpoints for creating and retrieving seed lists. These changes focused on improving the service layer and API usability.
scalableweb-crawlernutchosgiapache
USCDataScience/sparkler
May 2017 - Apr 2018
Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.
Role in this project:Back-end Developer
Back-end DeveloperContributions:27 commits, 4 PRs, 22 pushes in 10 months
Contributions summary:Sujen primarily contributed to enhancing the Sparkler crawler project. Their work focused on improving data handling and indexing within the Solr search engine, adding features such as storing fetched data segments and raw content from HTML pages. They also introduced a new generator to fetch children of high-scoring parents, and refined the codebase with bug fixes, configuration improvements, and logging adjustments.
pythonsolrweb-crawlernutchtika
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.