Junaid Effendi
Staff Data Engineer
New York, New York, United States
Summary
👤
Senior🎓
Top SchoolJunaid Effendi is a Staff Data Engineer in New York with 11 years building large-scale distributed data platforms and operationalizing pipeline reliability across batch and streaming systems. He has owned hundreds of pipelines and data models, redesigned streaming to meet 1-minute SLAs at 50K msgs/min, and led migrations and GovCloud (FedRAMP) projects while driving $25K/month cost savings through infra and pipeline optimizations. His toolkit spans AWS, GCP, Spark/Scala, Databricks, Airflow, Terraform, Snowflake and emerging GenAI tooling like LangChain, reflecting a rare blend of hands-on engineering and architecture. Prior roles include revamping data stacks to cut BigQuery costs by 70% and migrating large SQL codebases to Spark for 5x faster processing, demonstrating strong delivery and cost discipline. A former founder and early full‑stack engineer, he pairs product-minded instincts with technical leadership—mentoring teams, running tech talks, and pushing cross-team collaboration to accelerate data-driven decisions.
11 years of coding experience
13 years of employment as a software developer
Engineer’s Degree, Computer and Information Systems, Engineer’s Degree, Computer and Information Systems at NED University of Engineering and Technology
english, urdu
Stackoverflow
1reputation
73reached
0answers
2questions
Github Skills (64)
spark10
python10
r10
jdbc10
java10
scala10
scale10
sql10
hadoop9
apache9
apache-spark9
analytics9
large-scale9
metadata9
big-data9
Programming languages (7)
Github contributions (5)
MJFND/Web-Scraping-Visualization
Jun 2017 - Apr 2023
Contributions:19 pushes, 1 branch in 5 years 11 months
MJFND/R
Jun 2017 - Oct 2018
Contributions:138 commits, 136 pushes, 1 branch in 1 year 4 months
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.