Business Intelligence
Now Reading
Luigi
0
Review

Luigi

Overview
Synopsis

Luigi is a Python package that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization, handling failures, command line integration.

Category

Workflow Automation Software Free

Features

• Explicit dependencies
• No wasted time waiting
• Easy to tell what has failed
• Avoid duplicate work / partial failures

License

Proprietary

Price

Contact for Pricing

Pricing

Subscription

Free Trial

Available

Users Size

Small (<50 employees), Medium (50 to 1000 employees), Enterprise (>1001 employees)

Company

Luigi

What is best?

• Explicit dependencies
• No wasted time waiting
• Easy to tell what has failed

What are the benefits?

• Web interface too, so you can search and filter among all your tasks
•Visual overview of the dependency graph
•Build complex pipelines of batch jobs

PAT Rating™ ( Beta)
Editor Rating
Aggregated User Rating
Rate Here
Ease of use
7.9
8.4
Features & Functionality
7.8
9.0
Advanced Features
7.9
9.0
Integration
7.8
8.2
Performance
8.0
8.3
Customer Support
8.0
8.7
Implementation
8.8
Renew & Recommend
8.6
Bottom Line

Luigi is a Python package that helps users build complex pipelines of batch jobs, handles dependency resolution, workflow management, visualization, handling failures and command line integration.

7.9
Editor Rating
8.6
Aggregated User Rating
1 rating
You have rated this

Luigi is a Python package that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization, handling failures, command line integration.

Luigi has features such as visualiser page, dependency graph, task, scheduler, explicit dependencies between workflows, easier to write, vastly more extensible, code reuse and can easily re-run individual steps. Luigi is a user friendly plumber which gives simple dependency definitions with emphasis on Hadoop/HDFS integration, atomic file operations, data flow, visualization and command line integration.

Luigi has a central scheduler which ensures each task is only run by a single worker where a task is uniquely identified by its class name and its parameters. Luigi helps the users stitch many tasks together, where each task can be a Hive query, a Hadoop job in Java, a Spark job in Scala or Python, a Python snippet or dumping a table from a database.

Luigi is easy to build up long-running pipelines that comprise thousands of tasks and take days or weeks to complete where Luigi takes care of a lot of the workflow management so that the users can focus on the tasks themselves and their dependencies.

Luigi can build a pipeline that understands and respects the dependencies where Luigi alerts the users when a job fails and its dependencies aren’t run until the users fix the problem so it is easy for the users to resume the pipeline and have only incomplete jobs run for parallelizing jobs. Luigi also comes with a toolbox of several common task templates that the users use including support for running Python map, reduce jobs in Hadoop, as well as Hive, and Pig with file system abstractions for HDFS, and local files that ensures all file system operations are atomic.

You may like to read: Top Free, Premium & Open Source Workflow Automation Software and How to Select the Best Workflow Automation Software for Your Business

Filter reviews
User Ratings





User Company size



User role





User industry





Ease of use
Features & Functionality
Advanced Features
Integration
Performance
Customer Support
Implementation
Renew & Recommend

What's your reaction?
Love It
0%
Very Good
0%
INTERESTED
0%
COOL
0%
NOT BAD
0%
WHAT !
0%
HATE IT
0%