Charlie Lewis
Data Engineer
Dublin, Dublin 1, Ireland
Summary
🤩
RockstarCharlie Lewis is a Data Engineer based in Dublin with eight years of experience building and hardening production data pipelines. He specializes in orchestration with Airflow, having contributed to the popular ethereum-etl-airflow project by modernizing DAGs for Airflow 2, replacing deprecated operators, and improving temporary storage with GCS-Fuse. Comfortable across cloud storage and validation workflows, he focuses on reliable, efficient extraction and parsing of blockchain data into analytics platforms like BigQuery. Charlie brings a pragmatic approach to migrating and updating legacy pipelines while keeping downstream data quality in mind. His open-source contributions show a particular knack for adapting complex ETL systems to evolving platform ecosystems.
8 years of coding experience
Github Skills (14)
etl10
apache-airflow10
google-cloud-platform10
python10
ethereum10
data-engineering10
gcp10
data-pipelines9
data-pipeline9
bigquery9
docker8
dockers8
cicd7
google-cloud-storage7
Programming languages (6)
Github contributions (5)
blockchain-etl/ethereum-etl-airflow
Sep 2022 - Dec 2022
Airflow DAGs for exporting, loading, and parsing the Ethereum blockchain data. How to get any Ethereum smart contract into BigQuery https://towardsdatascience.com/how-to-get-any-ethereum-smart-contract-into-bigquery-in-8-mins-bab5db1fdeee
Role in this project:
Data Engineer
Data EngineerContributions:11 reviews, 40 commits, 22 PRs in 2 months
Contributions summary:Charlie primarily contributed to the development and maintenance of Airflow DAGs for extracting, loading, and parsing Ethereum blockchain data. Their work focused on adapting existing DAGs for Airflow 2, updating deprecated operators and hooks, and incorporating improvements such as GCS-Fuse for temporary storage. The commits demonstrate a focus on ensuring the data pipeline's functionality, validation, and efficiency within the context of the Ethereum blockchain data extraction and processing.
gcpethereumgoogle-cloud-platformcontractweb3
charlielewisme/charlielewisme
Nov 2020 - Nov 2020
Contributions:27 pushes, 3 branches in 1 day
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.