Arun Mahadeva Iyer
Sr. Staff Engineer at The Apache Software Foundation
California, United States
Summary
👤
Senior🎓
Top SchoolArun Mahadeva Iyer is a Sr. Staff Engineer based in California with 10 years of focused experience building large-scale distributed systems, real-time streaming platforms, and cloud-native big-data infrastructure. He currently drives Big Data infrastructure at Uber and has held principal engineering roles at Cloudera, Hortonworks and Yahoo, bringing deep production experience across AWS/Azure/Kubernetes environments. A long-time Apache committer and PMC member, Arun has contributed impactful fixes and performance improvements to flagship projects like Apache Spark, Storm and Kafka—work that includes optimizing Spark’s streaming readers and enhancing Storm’s windowing and SQL behaviors. He combines hands-on backend code contributions with systems-level architecture, helping teams transform streaming analytics into reliable, observable services. Colleagues rely on him for pragmatic solutions that balance low-latency processing, multi-threading correctness, and operational metrics at scale.
10 years of coding experience
14 years of employment as a software developer
B. Tech Computer Science, B. Tech Computer Science at Model Engineering College, Thrikkakara, CochinPre degree Mathematics and Science, Pre degree Mathematics and Science at Sacred heart college, Thevara, CochinRajagiri High School
English
Stackoverflow
368reputation
25kreached
12answers
3questions
Github Skills (16)
windowing10
javas10
big-data10
spark10
apache-storm10
sql10
java10
streaming10
scala10
kafka9
spark-structured-streaming6
apache-spark6
reliability6
zookeeper6
apache6
Programming languages (6)
Github contributions (5)
apache/storm
Jul 2016 - Feb 2019
Apache Storm
Role in this project:
Back-end Developer
Back-end DeveloperContributions:135 commits, 91 PRs, 723 comments in 2 years 7 months
Contributions summary:Arun primarily focused on enhancing the Apache Storm distributed streaming computation platform. Their contributions include resolving bugs related to windowing functionalities, such as correcting timestamp extraction in WatermarkCountEvictionPolicy and fixing group-by behavior in standalone mode. They also implemented the stream API and added support for sum functions with different types in storm-sql standalone mode. Additionally, the user improved code quality, addressed multi-threading issues, and fixed various minor bugs.
apachebig-datastormjavaapache-storm
apache/spark
Apr 2018 - Jun 2019
Apache Spark - A unified analytics engine for large-scale data processing
Role in this project:Back-end Developer
Back-end DeveloperContributions:14 PRs, 173 comments in 1 year 1 month
Contributions summary:Arun focused on improving the Apache Spark codebase, primarily addressing issues related to streaming functionalities. Their contributions include optimizing the `MemoryStreamDataReader` to avoid unnecessary sleeps, refactoring code to reflect the new `InputPartition` interface, and adding support for custom metrics reporting within the streaming query progress. These changes enhance the performance and provide better insights into the Spark's streaming job execution, specifically with the Kafka source.
analyticspythondata-processingsqlapache
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.