Daniel Walters
Data Engineer
United Kingdom
Summary
👤
Senior🎓
Top SchoolDaniel Walters is a data engineer with 11 years’ experience building scalable data pipelines and cloud infrastructure across GCP and AWS, currently driving large-scale projects at OnBuy such as an LLM-powered product categorisation of 18M SKUs and a Price Runner pipeline processing 7M+ products. He combines strong Python, Airflow and Terraform expertise with hands-on optimisation—e.g., cutting an ETL runtime from 8+ hours to under 2 via async Python and Cloud Run—and has led migrations that eliminated downtime for stakeholder reports. A chemist by training (First Class BSc, MPhil), Daniel brings a researcher's attention to data quality and automation, having implemented finance automations and GDPR-compliant data removal services. He also contributes to embedded C++/IoT open-source work (notably improvements to the Sming framework), reflecting a breadth from low-level firmware to cloud-native analytics.
11 years of coding experience
4 years of employment as a software developer
Master of Philosophy - MPhil Chemistry, Master of Philosophy - MPhil Chemistry at The University of ManchesterBachelor of Science - BSc Chemistry, Bachelor of Science - BSc Chemistry at University of Nottingham
Github Skills (16)
esp3210
app-framework10
web-framework10
iot10
embedded10
cpp10
arduino10
cplus10
esp826610
mqttnet9
lwip9
microcontroller9
mqtt9
rtc8
wifi7
Programming languages (15)
Github contributions (5)
SmingHub/Sming
Apr 2015 - Jul 2015
Sming - powerful open source framework simplifying the creation of embedded C++ applications.
Role in this project:
Embedded Systems Engineer / IoT Developer
Embedded Systems Engineer / IoT DeveloperContributions:22 commits, 3 PRs, 31 comments in 2 months
Contributions summary:Daniel primarily focused on enhancing the Sming framework for embedded C++ applications. Their contributions include implementing MQTT unsubscribe functionality, incorporating the use of `lwip` instead of `openlwip`, and fixing RTC-related issues. Furthermore, the user has addressed serial communication and WS2812 library integration. The commits demonstrate a clear understanding of the Sming framework and its underlying hardware.
rp2040c-plus-plussmingasyncssl-support
tavalin/smarthome
Feb 2016 - Feb 2018
Eclipse SmartHome project
Contributions:6 PRs, 18 pushes, 2 branches in 2 years
javaeclipsesmarthome
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.