Automated testing is a fundamental part of developing high-quality software. In CiviCRM, automated tests are divided into two categories: unit-tests and web-tests. Unit-tests run on the command-line and ensure the quality of CiviCRM APIs, functions, and classes. Web-tests run in the context of a simulated web browser, and are equivalent to clicking on buttons and filling out fields – web-tests ensure that the pieces of CiviCRM fit together to form a cohesive system. Both unit-tests and web-tests build on top of PHPUnit.
Web tests are developed using a tool called "Selenium", so the terms "web test" and "Selenium test" refer to the same thing: tests which either run within the context of the browser (Selenium IDE) or simulate the browser's actions at the command line (Selenium Server).
In software engineering, web tests may sometimes be called "integration tests".
Populate the database schema and metadata by loading these two files: "civicrm/sql/civicrm.mysql" and "civicrm/sql/civicrm_generated.mysql"
cd /path-to/civicrm echo "create database civicrm_tests_dev" | mysql -u user -p -hhostname mysql -u user -p -hhostname civicrm_tests_dev < sql/civicrm.mysql mysql -u user -p -hhostname civicrm_tests_dev < sql/civicrm_generated.mysql
It is helpful but not strictly necessary to name the database "civicrm_tests_dev" – subsequent steps will be easier if you use this default name. However, if you are an advanced developer who works concurrently on multiple versions of CiviCRM, then you may want to create separate databases with different names..
When executing unit-tests on the command-line, the tests will need to connect to the MySQL database – by providing a username, password, hostname, and password. There are several options for configuring the database connection. (Note: In all cases, the user must have "SUPER" privileges.) Choose any one of the following:
Configure the MySQL configuration file (~/.my.cnf) with a username and password. This is the simplest approach, but it only works if the database is called "civicrm_tests_dev".
Creating a ~/.my.cnf configuration file in your home directory will allow you to run unit-tests as well as MySQL command-line tools without re-typing the username and password. For example, put the username and password in the file using this format:
Note that 'host' defaults to 'localhost' and 'user' defaults to $USER (your logged-in username).
Configure the PHP constant CIVICRM_DSN.
To define PHP constants for unit-testing, copy the file "<path to civicrm>/tests/phpunit/CiviTest/civicrm.settings.local.php.txt" to "<path to civicrm>/tests/phpunit/CiviTest/civicrm.settings.local.php". Edit the file to include to include a line like this:
To run a single test – for example, "api_v3_ContactTest" – execute following command:
cd /path-to/civicrm/tools scripts/phpunit api_v3_ContactTest
If you have chosen to pass the MySQL connection as parameters (rather than putting the connection info in a file), then pass user/password/host/database like so:
cd /path-to/civicrm/tools scripts/phpunit -udb_username -pdb_password -hdb_host -bdb_name api_v3_ContactTest ## Notes: ## * There is no space between -u, -p, -h, -b and the corresponding parameters. ## * The db_username must have super privileges. ## * On Windows you will get errors about "stty", but it works anyway.
You may get unexpected errors or warnings if using the wrong version of PHP. For example, OS X users who run MAMP may receive warnings about timezone settings. This happens because the system includes multiple copies of PHP – one distributed with OS X (which is poorly configured/built), and a better version distributed with MAMP (which is better configured/built). To address this, see Setup Command-Line PHP.
To run the whole suite of tests, change the word "api_v3_ContactTest" to "AllTests". The full suite of tests can take a number of hours to complete, so it may be best to run this overnight.
cd /path-to/civicrm/tools scripts/phpunit AllTests
To run a single test (one method in the test class), add the --filter parameter.
cd /path-to/civicrm/tools scripts/phpunit --filter test_method_name$ api_v3_ContactTest
--filter is evaluated as a regular expression and compared to test method names.
EXAMPLE: To run ONLY the "testCreateEmptyContact" method:
Sometimes, when you're running your tests, phpunit might quit silently and it's hard to identify which specific test does it. Please use --tap parameter in such cases, it will give you more descriptive output, e.g.
To run unit tests for an extension, see the relevant documentation.
The process of using web-tests is generally the same as using unit-tests – one must create a test database, create a database connection, and run "phpunit" commands. However, there is a key difference: while the unit-tests perform all data operations on an isolated database (civicrm_tests_dev), the web-tests require a fully-functioning CiviCRM-Drupal installation, and they require extra information for simulating web-requests. To configure these, setup CiviSeleniumSettings.php:
cd /path-to/civicrm/tests/phpunit/CiviTest cp CiviSeleniumSettings.php.txt CiviSeleniumSettings.php
Edit the civicrm.settings.local.php that was created in the PHPUnit setup and add or adjust CIVICRM_TEMPLATE_COMPILEDIR to a full path that is writable for the user who will be running tests (note that using "~" in the path may not work - use full paths).
If you want to run Selenium tests, go totests page for details.