Sign in to see all reviews and comparisons. It's Free!
By clicking Sign In with Social Media, you agree to let PAT RESEARCH store, use and/or disclose your Social Media profile and email address in accordance with the PAT RESEARCH Privacy Policy and agree to the Terms of Use.
Luigi is a Python package that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization, handling failures, command line integration.
Category
Workflow Automation Software Free
Features
• Explicit dependencies • No wasted time waiting • Easy to tell what has failed • Avoid duplicate work / partial failures
License
Proprietary
Price
Contact for Pricing
Pricing
Subscription
Free Trial
Available
Users Size
Small (<50 employees), Medium (50 to 1000 employees), Enterprise (>1001 employees)
Company
Luigi
What is best?
• Explicit dependencies • No wasted time waiting • Easy to tell what has failed
What are the benefits?
• Web interface too, so you can search and filter among all your tasks •Visual overview of the dependency graph •Build complex pipelines of batch jobs
PAT Rating™
Editor Rating
Aggregated User Rating
Rate Here
Ease of use
7.9
6.7
Features & Functionality
7.8
7.3
Advanced Features
7.9
9.0
Integration
7.8
8.2
Performance
8.0
7.7
Customer Support
8.0
8.7
Implementation
8.8
Renew & Recommend
8.6
Bottom Line
Luigi is a Python package that helps users build complex pipelines of batch jobs, handles dependency resolution, workflow management, visualization, handling failures and command line integration.
7.9
Editor Rating
8.1
Aggregated User Rating
3 ratings
You have rated this
Luigi is a Python package that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization, handling failures, command line integration.
Luigi has features such as visualiser page, dependency graph, task, scheduler, explicit dependencies between workflows, easier to write, vastly more extensible, code reuse and can easily re-run individual steps. Luigi is a user friendly plumber which gives simple dependency definitions with emphasis on Hadoop/HDFS integration, atomic file operations, data flow, visualization and command line integration.
Luigi has a central scheduler which ensures each task is only run by a single worker where a task is uniquely identified by its class name and its parameters. Luigi helps the users stitch many tasks together, where each task can be a Hive query, a Hadoop job in Java, a Spark job in Scala or Python, a Python snippet or dumping a table from a database.
Luigi is easy to build up long-running pipelines that comprise thousands of tasks and take days or weeks to complete where Luigi takes care of a lot of the workflow management so that the users can focus on the tasks themselves and their dependencies.
Luigi can build a pipeline that understands and respects the dependencies where Luigi alerts the users when a job fails and its dependencies aren’t run until the users fix the problem so it is easy for the users to resume the pipeline and have only incomplete jobs run for parallelizing jobs. Luigi also comes with a toolbox of several common task templates that the users use including support for running Python map, reduce jobs in Hadoop, as well as Hive, and Pig with file system abstractions for HDFS, and local files that ensures all file system operations are atomic.
We are the movers and shakers of B2B Software & Services. We have been copied by many including the you know who?
Why not get it straight and right from the original source. Join over 66,000+ Executives by subscribing to our newsletter... its FREE ! and get fully confidential personalized recommendations for your software and services search
Privacy Policy: We hate SPAM and promise to keep your email address safe.
By clicking Sign In with Social Media, you agree to let PAT RESEARCH store, use and/or disclose your Social Media profile and email address in accordance with the PAT RESEARCH Privacy Policy and agree to the Terms of Use.