Documentation/Scheduling/Batch Schedules and Jobs

Documentation :: Scheduling :: Batch Schedules

Batch Schedules allow multiple scripts to be executed in a specific order at predefined times. This can be useful when certain scripts depends on the successful completion of another, especially when these scripts are distributed across a number of hosts.

Batch Schedule Jobs can be have any number of dependencies on other scripts being completed, and additionally be set to run at specific times or on the presence, absence or content of a file.

Example

This example tries to demonstrate a simple use case of using a Batch Scheduled Script. In this example we have three scripts:

- housekeeping.sh
- ETL.sh
- cacheClear.sh [website]

Every night, we need to run housekeeping.sh, followed by ETL.sh, and finally cacheClear.sh twice with different arguments. This would give a schedule flow that looks something like below.

Example Batch Schedule

                                                 ==============================
                                                 +                            +
                                           -->   + cacheClear.sh website1.com +
===================         ============         +          (host 1)          +
+                 +         +          +         ==============================
+ housekeeping.sh +   -->   +  ETL.sh  +      
+    (host 1)     +         + (host 2) +         ==============================
===================         ============         +                            +
                                           -->   + cacheClear.sh website2.com +
                                                 +          (host 1)          +
                                                 ==============================

This schedule ensures that ETL.sh is not run before housekeeping.sh is finished, and both runs of cacheClear.sh do not run until ETL.sh is completed, and enables both cacheClear.sh runs to happen in parallel. All of this can easily happen despite these scripts running on different hosts.

To do this, first we need to create the Batch Schedule to run this under. From the menu, select Schedules > Batch Schedules > Manage Batch Schedules, and then click .



We can fill in this schedule as shown above.

Once this is created, we can then add our scripts, referred to as Batch Schedule Job. From the menu, select Schedules > Batch Schedules > Manage Batch Schedule Jobs, and click .



We can create the first job as shown above. We have set this to run at 23:00 every night. Once this is created, we can then add the second job.



As shown above, this time we need to set the first script as a predecessor job of this script. This is what ensures that ETL.sh does not start until housekeeping.sh has successfully completed. The Allow Failures option allows ETL.sh to run even if housekeeping.sh reports it failed.