Skip to end of metadata
Go to start of metadata


CiviCRM Documentation


Developer Resources


CiviCRM books!

Make sure to check out our Online User/Administrator Guide!

Or support us by buying an eBook or hard copy of Using CiviCRM and The CiviCRM Cookbook from Packt Publishing.

Overview

CiviCRM includes a number of "Jobs" which process data and handle batch tasks "in the background". The available jobs are listed below, with a description of their purpose and required parameters. Your site / system administrator should review the available jobs, and determine which ones need to be run on a regular basis. These jobs can be configured and enabled from the Scheduled Jobs page (Administer > System Settings > Scheduled Jobs). Jobs can also be executed manually from this page. However, for most sites it is best to run the required jobs from the server command line, often as one or more automatically scheduled "cron" jobs.

A "run frequency" can be set for each enabled job, allowing you to configure a single "master" cron task to trigger ALL enabled CiviCRM jobs whose next run is due. For example, you may want to run the CiviMail mailing scheduler every time your master cron task runs (which might be once a minute). However, you only want the Address geocoder job to run once a day. You can enable both jobs, and set the run frequency for the mailing scheduler to "Every time cron job is run", while setting the frequency for the Address geocoded to "Daily".

The format to enter the parameters for those jobs that have available parameters listed in the table below is one parameter per line. For example for Greetings Updater you can enter the following in the parameters textbox to update all Individual postal greetings:

To actually trigger all the enabled jobs in Scheduled Jobs you will have to set up a cron job on the server that does this. You can find out how below in the section "Command-line Syntax for Running Jobs".

Note that the parameters for the relevant scheduled jobs should be returned by running getfields on the relevant job. We really want the UI to leverage these getfields results for dynamic documentation

Scheduled Jobs

Component

Job Title (API action)

Description

Parameters

Deprecated Bin Script

Core and CiviCase

Activity Processor/Process Inbound Emails (fetch_activities)

Inserts activities for a contact or a case by retrieving inbound emails from a mail directory

(none)

Email2Activity.php

Core

Geocode and Parse Addresses (geocode)

Retrieves geocodes (lat and long) and/or parses street addresses (populates street number, street name, etc.) in bulk. Geocoding provider must be configured (Administer > System Settings > Mapping and Geocoding). Street Address Parsing must be enabled (Administer > Localization > Address Settings).

geocoding=1 (0 to skip geocoding)
parse=1 (0 to skip address parsing)
start=null (optional, begin processing with this contact ID)
end=null (optional, process contacts with ID's <= this value)
throttle=0 (1 to add a 5 second sleep between each geocoding request)

UpdateAddress.php

Core

Clean-up Temporary Data and Files (cleanup)

Removes temporary data and files, and clears old data from cache tables. Recommend running this job every hour to help prevent database and file system bloat.

(none)

 

CoreDisable expired relationships (disable_expired_relationships)Disables relationships that have expired (ie. those relationships whose end date is in the past).(none) 

Core

Greetings Updater (update_greeting)

Updates Email Greeting, Postal Greeting and / or Postal Addressee for contacts based on a passed in format or your site defaults found in Administer > Communications > Email Greeting Formats and Administer > Communications > Postal Greeting Formats.

This job is disabled by default and can not be enabled to run automatically as part of a scheduled jobs run. It can only be run manually (single job) from the Scheduled Jobs page or command line.

(learn more...)

ct= (required: Individual, Organization, or Household)
gt= (required: email_greeting, postal_greeting, or addressee)
id= (optional: force script to set a greeting or addressee format other than default for a given contact type, use ID of format option value)
force=0 (1 to update all contacts, otherwise only contacts with null values are updated)

UpdateGreeting.php

Core

Mail Reports (mail_report)

Generates and sends a copy of the specified report instance to the email addresses configured in that instances Report Settings.

instanceId= (required, ID of report instance to send)
format=csv (optional, to output the report as a CSV file instead of default PDF format)
format=print (optional, to output the report as printer-friendly HTML)
sendmail=0 (optional, tells the job NOT to email the report. use this in combination with print or csv format to write report to stdout so you can save it to a disk file)

CiviReportMail.php

CoreRebuild Smart Group Cache (group_rebuild)Rebuilds the smart group cache.

limit=Number optional-Limit the number of smart groups rebuild

 
Core(?)Send Scheduled SMS (process_sms)Sends out scheduled SMS(none) 

Core and CiviEvent

Send Scheduled Reminders (send_reminder)

Sends out scheduled reminders related to events or activities via email

(none)

action.cronjob.php

CiviCampaign

Process Survey Respondents (process_respondent)

Releases reserved survey respondents when they have been reserved for longer than the "Release Frequency" days specified for that survey.

(none)

RespondentProcessor.php

CiviEvent

Update Participant Statuses (process_participant)

Updates participant statuses for Wait-list and Approval Required event registration features. Moves participants from "On waitlist" to "Pending from waitlist" as space becomes available. Moves any type of Pending registrations to "Expired" if expiration period is set for an event. Sends notifications of status changes to participants.

(none)

ParticipantProcessor.php

CiviMail

Mailings scheduler (process_mailing)

Sends queued/scheduled CiviMail mailings

(none)

civimail.cronjob.php

CiviMailFetch Bounces (fetch_bounces)Processes bounced email from the return channel. Fetches bounces from mailings and writes them to mailing statistics.(none)EmailProcessor.php
CiviMailValidate Email Address from Mailing (update_email_resetdate)Updates the reset_date on an email address to indicate that there was a valid delivery to this email address.

minDays, maxDays=Consider mailings that have completed between minDays and maxDays

 

CiviMember

Update Membership Statuses and Send Renewal Reminders (process_membership)

Updates membership statuses and optionally sends renewal reminders

When renewal reminders are sent, the member's reminder date field will be set to NULL in the database and on screen. When the membership is renewed, the field is populated with the updated renewal reminder date.

 

UpdateMembershipRecord.php

CiviMember

Set Renewal Reminder Dates (process_membership_reminder_date)

Sets membership renewal reminder dates for current membership records where reminder date is null.

This job should never be run automatically as it will cause members to get renewal reminders repeatedly. It is disabled by default to prevent it from being included in an "run all jobs" session.

(none)

UpdateMembershipReminderDate.php

CiviPledge

Process Pledges (process_pledge)

Updates pledge payment and pledge statuses and optionally sends payment reminders

send_reminders=1 (optional: add this parameter if you want pledge payment reminder emails sent)

UpdatePledgeRecord.php

Scheduled Jobs Editing Fields

Creating a Scheduled Job requires filling in API fields that are similar to those used in the API Explorer.  Find out more about using the API here and detailed instructions (developer oriented) here.

The API fields are:

  • Field 1: API - civicrm_api3 (occasionally you may need to use civicrm_api2)
  • Field 2: API name - this is a drop-down list and corresponds to the filename of files in the civicrm/api/v3 directory, ie, Activity in the list corresponds to file Activity.php in the directory.
  • Field 3: Action - typically get, create, getvalue, update, delete, etc.  See the list of typical actions in the documentation, but note that each API may define its own actions.  Documentation for APIs and actions is here, but you may need to examine the files in the civicrm/api/v3 directory for the most detailed information available about APIs, actions, and options.  
  • Field 4: Command parameters -  This is a list of parameters and values, like

entity_id=102
contact_id=12932
limit=10
throttle=1
list=0
type=text

Again you'll need to read the API reference or check the documentation in the files themselves for details about available command parameters and their effects.

The result of this entry is a scheduled call to the API function defined by the fields.  For example, if your fields were:

  civicrm_api3 Activity get  with parameter contact_id=352

The result would be a call to function civicrm_api3_activity_get ($params) in file civicrm/api3/Activity.php, where $params=array(contact_id->532).

You can explore APIs, actions, and parameters at the API explorer, which is included in your CiviCRM installation at yourwebsite.com/civicrm/api/explorer. An explanation and video tour of the API explorer is here. Once you have a working API call at the API explorer you can transfer the same API call, action, and parameters to create a new Scheduled Job that will carry out that same task automatically.

 

Command-line Syntax for Running Jobs

Drush method

/usr/bin/drush -u 1 -r /path/to/drupal @sites civicrm-api job.execute auth=0 -y

(-u 1 means user 1, put the user that has the needed permissions, with 1 you're sure it will work)

or

/usr/bin/drush -u 1 -r /path/to/drupal -l www.example.org civicrm-api job.execute auth=0 -y

See the drush documentation for the -r and -l options. If successful, this command will output an array with a key/value of error = 0. In addition to the Drupal user permissions, you may also need to pay attention to the file permission of the linux user that the command runs as (e.g. typically run this command as the same user as apache - "apache", "nobody", or "www-data" are the most likely candidates).

PHP cli method

This method requires a valid Username and Password (for a Drupal, Joomla or WordPress user who has adequate permissions for the job or jobs being run). If you use drupal, use the drush method, that is easier to configure and more secure (it doesn't require to put the user password)

Typical set up: Use Cron to give CiviCRM the chance to process enabled scheduled jobs according to the frequencies set. e.g. every 15 minutes is a reasonable poll time. Here's a template line you might use in your user crontab:

The -s parameter is the site name, and defaults to 'localhost'. It should be set the FQDN (fully qualified domain name) of your site (eg, 'example.com') Some jobs rely on this being set (in certain cases, the CiviMail mailing scheduler uses this FQDN to generate absolute URLs when relative URLs are in the email).

To run ALL scheduled jobs that are Enabled and are due to run based on last run and run frequency as a one-off from the command-line:

To run a specific job (regardless of Enabled status or last run time):

For example, to run Greeting updater job which requires ct (contact type) and gt (greeting type) parameters

Enable Logging with cli.php for single jobs

Entries are ALWAYS written to the job log when you use 'execute' action to run all scheduled jobs. Single jobs run via cli.php do not write to the Job Log by default. If you want a single job run to be logged, add -j to the beginning of the command line.

Make sure there that there are no reserved or unsafe URL characters in your username or password. These include:

and spaces.

URL method

You can run ALL scheduled jobs that are Enabled and are due to run based on last run and run frequency by invoking a URL with curl or wget. You can also invoke specific jobs via URL. This method requires a valid Username and Password (for a Drupal, Joomla or WordPress user who has adequate permissions for the job or jobs being run). It also requires you to pass in the CIVICRM_SITE_KEY which is configured by editing your copy of civicrm.settings.php (see Configuring your Site Key below).

More Info on Using WGET or CURL

Running Command-line Scripts via URL

Logging with URL Method

Any jobs that are invoked using the URL method are automatically logged.

Run ALL Scheduled Jobs via URL

Calling cron.php as shown below will run ALL scheduled jobs that are Enabled and are due to run based on last run and run frequency:

In Drupal:

In Joomla:

In WordPress:

 

When calling these urls using wget from within a crontab, enclose the url in single quotes, like so:

An alternative way to set up cron job is using webcron like https://www.easycron.com.

 

Run a Specific Job via URL

You can also run any single job via URL by passing job=<api_action> as well as any required parameters. For example to run the Update Greeting job via URL:

In Drupal:

In Joomla:

In WordPress:

Specific Job via URL works only for API entity = job

Note that you can execute API actions only from the API entity "job"; this is hard-coded into cron.php.  So you can't call any of the built in API functions (other than those under entity "job") using cron.php, but you also can't create a new API action  So for instance, if you create new API entity called "myapi" with API action "my action" then you would be able to call that action via the Scheduled Jobs page but you will not be able to call it via cron.php.

A possible workaround for this problem is here: http://forum.civicrm.org/index.php/topic,28215.0.html

Running CiviCRM crons in Aegir

If you are using the Aegir hosting environment, with the provision_civicrm sub-module, you can automate the installation of a new cron for each site using the hosting_civicrm_cron module. You can also use the Drush method below to call CiviCRM cron tasks.

Running CiviCRM crons using Drush

You can run cron manually for a site by using the drush command:

This assumes that you have an administrator account called "admin" with the correct CiviCRM permissions.

Currently (CiviCRM 4.2.x) you need to use an additional argument, 'auth=0', to work around CRM-11199.

Configuring your Site Key

If you are triggering jobs using the URL method, you must manually generate a site key, and then add it to your CiviCRM settings file.

  • First generate the key. We recommend using a 16-32 bit alphanumeric/punctuation key value. You can ...
    • create a random 16-32 character key from your keyboard OR
    • use an external service like: http://www.thebitmill.com/tools/password.html OR
    • use the following shell command to generate a key

      Make sure there that there are no reserved or unsafe URL characters in your site key. These include:

      and spaces.

  • Open your settings file - civicrm.settings.php - in your favorite editor and either add or edit the CIVICRM_SITE_KEY value.

in Drupal, civicrm.settings.php is located in /sites/default

in Joomla, there are 2 civicrm.settings.php files.  By default these are located in site/administrator/components/com_civicrm and site/components/com_civicrm - it's important to add the same key to both files as some Joomla front-end functions rely on the key (eg, decrypting the Outbound Mail SMTP Password)

in WordPress, civicrm.settings.php is located in /wp-content/plugins/civicrm

Depending on which version of CiviCRM you've installed, you may have this setting in your file already with a value of null, or you may need to add the line. In either case, replace null with the value of your key.

EXAMPLE: If my generated site key value is: 3cx4aNkpQwxmM1hTMV~!B09iO6
... then my settings file should look like this:

* Finally, make sure that you include the key=<sitekey> parameter whenever your run any of these scripts.

Troubleshooting

DB Error: Connect failed

The connect failed error message means that CiviCRM was not able to connect to the SQL database at all.  The problem may be in connecting to the CiviCRM database OR with the Drupal/Joomla/WordPress database.  If your CiviCRM installation is otherwise working, the most likely cause is the connection to the Drupal/Joomla/WordPress database, which is accessed much less frequently.  

The most likely cause is incorrect username, password, or other settings in the civicrm.settings.php file. Carefully check the setting for define( 'CIVICRM_UF_DSN' - your setting may have been incorrect for some time and enabling the cron process is only now exposing the problem.

DB Error: no such table

A common cause of this problem is if your Drupal installation has a table prefix. To validate the connection, CiviCRM looks at the users table while initializing the cron job.  The specific name of the users table is set in yourpath/civicrm/admin/setting/uf (or civicrm/administer/system settings/CMS database integration).  If the users table name set there does not match the actual table name in your Drupal SQL database you will get the 'no such table' error.  Check your Drupal table prefix in settings.php (usually in sites/default) - look for a setting like $db_prefix = 'dru_';. In yourpath/civicrm/admin/setting/uf change the "Drupal Users Table Name" to match–for instance, if your prefix is dru_, change the table name to dru_users.

More information on this issue is here - check the comments.

Wrong permissions for cli.php

If you are running cli.php directly via cron job you may need to change permissions of cli.php to allow it to execute.  Depending on how cron runs and the owner of cli.php you might try one of these options–or use your ftp program to change file permission for cli.php similarly:

chmod 744 cli.php                [Gives owner execute permissions]
chmod 754 cli.php                [Gives owner & group execute permissions]
chmod 755 cli.php                [Gives everyone execute permissions]

Running cli.php via php (as recommended above) will also solve this problem.  Use this:

php /your/directory/structure/sites/all/modules/civicrm/bin/cli.php -u user -p password -e Job -a execute

Rather than this:

/your/directory/structure/sites/all/modules/civicrm/bin/cli.php -u user -p password -e Job -a execute

Also, some systems will require the cli version (e.g. /usr/bin/php-cli vs /usr/bin/php)

Using the URL method with https and cron fails silently

If your server has an older version of wget installed and you are trying to connect via https there are a couple of problems that can cause wget to exit.

The solution is to upgrade wget to a version with the proper https support. A workaround is to use the flag --no-check-certificate in your wget call.

*/15 * * * * wget --no-check-certificate -O - -q -t 1 'https://mysite.org/sites/all/modules/civicrm/bin/cron.php?name=username&pass=password&key=site-key'

Other issues

Two threads in the CiviCRM Forums have ideas for troubleshooting problems with the scheduled jobs.  If you find specific problems or solutions, please make note of them here.

If you've just upgraded to CiviCRM 4.3 on a multi-site install and your cron silently stops working, try switching to the Drush method (forum thread).

If you're using HostGator, and switched to PHP 5.3, check this thread.

Labels
  • None