Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
Contributions:35 pushes, 9 branches in 6 years 4 months
dependency-resolutionhadooppython-modulesvisualization
Contributions:47 commits, 244 pushes, 24 branches in 3 years 8 months