Andy Jackson
Digital Preservation Specialist
Greater Leeds Area United Kingdom
Summary
🤩
Rockstar🎓
Top SchoolAndy Jackson is a Digital Preservation Specialist with 15 years' experience designing and implementing large-scale web archiving and preservation systems, currently shaping registries and policy at the Digital Preservation Coalition. With a PhD in Computational Physics and a background in massively-parallel scientific computing, he brings rigorous modeling and performance-minded architecture to web-scale ingestion, indexing and replay pipelines. He led the British Library's technical team for Legal Deposit web archiving and contributes to core open-source projects such as pywb and Heritrix, where his backend fixes and configurable enhancements improve reliability and scalability. Comfortable from mobile clients to supercomputers, he excels at clarifying technical debates and turning preservation principles into pragmatic, auditable systems. Known among peers as an “entropy buster” and partial physicist, he pairs curiosity with meticulous debugging to tackle tricky resource and indexing issues behind the scenes.
15 years of coding experience
20 years of employment as a software developer
Ph.D. Computational Physics, Ph.D. Computational Physics at The University of EdinburghMPhys Computational Physics, MPhys Computational Physics at University of York
English
Stackoverflow
356reputation
139kreached
10answers
0questions
Github Skills (31)
debugging10
wayback-machine10
web-crawler10
archiving10
webscraper10
python10
amqp10
archives10
webscraping10
mergetool10
java10
crawling10
webar10
javas10
heritrix10
Programming languages (23)
Github contributions (5)
iipc/openwayback
Oct 2013 - Mar 2017
The OpenWayback Development
Role in this project:
Back-end Developer
Back-end DeveloperContributions:126 commits, 14 PRs, 4 pushes in 3 years 5 months
Contributions summary:Andy primarily focused on resolving merge conflicts within the project, specifically addressing issues in `pom.xml` and `wayback-core/pom.xml` related to the integration of updates. Their contributions also included modifications to the `EmbeddedCDXServerIndex.java` file, indicating work on the core indexing functionality of the Wayback Machine. Furthermore, the user addressed resource leaks in the `ResourceFactory.java` file and resolved conflicts in the `FastArchivalUrlReplayParseEventHandlerTest.java` file, showcasing debugging and testing efforts.
internetarchive/heritrix3
Sep 2015 - Jul 2022
Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.
Role in this project:Back-end Developer
Back-end DeveloperContributions:11 releases, 5 reviews, 196 commits in 6 years 11 months
Contributions summary:Andy primarily contributed to the `AMQPUrlReceiver` component within the Heritrix3 project. They implemented configurable features for the AMQP receiver, including settings for queue durability, auto-delete, and prefetch counts. Key changes involved separating the consuming status from the running status and integrating functionalities for seeds. Further modifications included adding support for header modifications in the scheduling process and bumping priority levels.
heritrixweb-crawlerjavawebcrawlingwarc
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.