Consolidated CiviCRM Cron

Documentation for this functionality can be found at Managing Scheduled Jobs

Some remarks from conversations/discussions about this spec that already happened: don't use Cron Expression, just provide ability to configure job to run once a day or every time the "base job" invoking other cron jobs is run being able to cron jobs run through both web and cli we need to address timeouts issue somehow administrators should be able to goto a page and see what cron job ran and when and what was the output (if any) - included each cron job should be implementing simple interface (or rather abstract class), containing at least 4 methods: run - used to kickstart the job, potential "run piece by piece" logic implemented here if needed. This method is cli/web agnostic and gets all it parameters from the base class / environment logStart - put the log entry that cron job started at given time(can be implemented in abstract class) logEnd - put the log entry that cron job finished at given time, plus additional information about success/failure (can be implemented in abstract class) logEntry - put log entry during cron exection integrate with hook_cron drupal (already included) remove the need for username and password in web script calls, introduce a setting for default contact (id)

This page refers to Consolidated CiviCRM Cron MIH and describes the concept of implementation of this feature.

Basic concept

CiviCRM will have single cron script/url (depending on user preference), which will need to be configured in "traditional way" - by adding a line to server's crontab. Recommended configuration for this cron job is to be run every minute for best accuracy or at least every hour for acceptable accuracy.

This script/url will trigger a check on Scheduled Jobs list configured inside CiviCRM's administration panel and stored in CiviCRM database. It will execute those based on their individual "cron like" configuration. All the Scheduled Jobs runs are logged - both their start and end. To make transition between current way of handling things and new way, in this phase, Scheduled Job is an equivalent to existing script (e.g. UpdateMembershipRecord.php) and is executed by "emulating" url call method.

"Scheduled Job" name is used to avoid confusion with cron job (which is required to run Scheduled Jobs).

Permissions

"Administer CiviCRM" permission is required to manage Scheduled Jobs.

Schema changes

Two tables will be added: civicrm_job, to store information about scheduled jobs, and civicrm_job_log, to save job run information.

User interface

Administer -> Manage -> Scheduled Jobs (admin/job)

Default view

This screen presents list of configured Scheduled Jobs, allows going to edit job screen, as well as disabling/enabling them and deleting them. Also contains a link to Add Job screen.

Add action

Each scheduled job has following properties, that can be configured by administrator:

• Name - required, unique name of the Job
• Description - optional description
• Cron String - required, cron string stating the frequency of runs (e.g. "5 * * * *" )
• Script - required, file name of the script. Url is being constructed using this name, assumed directory is $civicrm_root/bin, assumed base URL is "CiviCRM Resource URL" from global settings.

Administer -> Manage -> Scheduled Jobs Log (admin/joblog)

This screen presents the list of etries from civicrm_job_log screen. The only action that can be done on this screen is purging the log (effecting in DELETE from civicrm_job_log).

Other functionality

• civicrm_job_log should be automatically purged (DELETE FROM civicrm_job_log WHERE run_datetime < 7 days ago) on every scheduled jobs run.

Implementation notes

• In order to parse and interpret Cron String property of Scheduled Job, Cron Expression library is going to be used. It offers parsing cron expressions and checking if given cron expression makes specific job due to run relative to current moment in time. Every time "main cron job" is run, all the Scheduled Jobs are checked whether they're due, and run or skipped according to test result. NOTE: exact accuracy of cron expressions defined for Scheduled Jobs can be achieved only if "main cron job" is set to (* * * * * - run every minute), therefore this should be recommended setting.
• CRM_Utils_System::authenticateScript will need to be modified to provide authentication based on "short living" hash string generated for each cron script before it's run and stored in civicrm_job. Hash string is passed as parameter in url and authenticateScript checks whether parameter matches the contents of database before it authenticates the script.

Second phase - potential tasks and improvements

• implementing Drupal cron hook, so it's not necessary to configure separate cron job just for CiviCRM.
• implementing user friendly interface to replace Cron String
• designing CiviCRM Scheduled Job API - each script (to be specific, PHP class) will need to have specific methods, e.g. will have to implement "class->run() method, class->resultLog() method for storing run log message (success/failure), etc
• reworking existing scripts to follow above "API"
• adding alternative job call method - "callback" pointing to specific PHP class