Azkaban

Azkaban is a robust server solution which was implemented at LinkedIn to solve the problem of Hadoop job dependencies that needed to run in order, from ETL jobs to data analytics with its 3 key components like the Relational Database (MySQL), Azkaban Web Server and Azkaban Executor Server. Azkaban has features such as project management, executing flow state, previous flow and jobs, scheduler and SLA.

The Azkaban Web Server is the main manager to all of Azkaban which handles project management, authentication, scheduler, and monitoring of executions and also serves as the web user interface. Azkaban uses *.job key-value property files to define individual tasks in a work flow, and the _dependencies_ property to define the dependency chain of the jobs where job files and associated code can be archived into a *.zip and uploaded through the web server through the Azkaban UI or through curl.

Azkaban will validate the contents of the zip to make sure that dependencies are met and that there's no cyclical dependencies detected and if it finds any invalid flows, the upload will fail because uploads overwrite all files in the project so any changes made to jobs will be wiped out after a new zip file is uploaded. When a successful upload is made in Azkaban, the users should see all of their flows listed on the screen.

Azkaban lets users create projects where the creator is automatically given an ADMIN status on the project which allows the creator to view, upload, change jobs, run flows, delete and add user permissions to the project. An admin can also remove other admins, but cannot remove themselves to prevent projects from being admin-less except when admins are deleted by a user with an admin role.

You may like to read: Top Free, Premium & Open Source Workflow Automation Software and How to Select the Best Workflow Automation Software for Your Business