"Perfect is often the enemy of the Good, or the Good Enough."
Ideally, we would have a long list of automated tests that each Civi release could be tested with. While there are some automated tests and work is proceeding on improving those tools, they are not perfect and of themselves they don't get the job done. This document is an attempt to build a list of items that we can manually test each RC with by splitting the workload and working together. Once the list is built, we can look at dividing the labor associated with who will own which items for each RC.
If we did each test for all three CMS’s, each line item would have wither three iterations or six depending on whether or not there was a concurrent LTS release.
This page is meant specifically for creating the list of the tests. For implementation, the thought is that we would create a spreadsheet of this list with columns for all three CMS's in the most recent release and the LTS release.
That spreadsheet was made: https://docs.google.com/spreadsheets/d/1UgH36o9fMlTSX_ZTg38LdtfkPDx2RN9MFYvuiPfeJIk/edit#gid=0 but then interest was lost it seems....
- How should we test permissions?
- Can folks who are familiar with the existing automated tests suggest in which areas deficiencies in the automated testing would make manual testing exceptionally valuable?
- Add a contact
- Edit a contact
- Delete a contact
- Create a contact sub-type
- Add a contact of that sub-type
- Modify a contact of that sub-type
- Delete a contact of that sub-type
Import / Export
Import a CSV of contacts
Export a CSV of contacts
Custom Field Sets
- Add and Edit: Custom Data Fieldset With Following Scopes
- Contact Subtype(s)
- Event Type(s)
- Event Name(s)
- Financial Type(s)
- Participant Role(s)
- Try Advanced Search Using Any or All of the Above Custom Data
- Create an event
- Edit an event
- Register a participant in the back end
- Register a participant on the front end
- Edit a participant
- Make a contribution on the back end
- Edit a contribution on the back end
- Make a contribution on the front end
- Make a recurring contribution on the front end
- Edit a recurring contribution in the member dashboard
- Sign up for a membership on the front end
- Add a membership on the back end
- Cancel a membership on the back end
- Basic Search by name
- Basic Search by type
- Basic Search by group
- Basic Search by tag
- Basic Search by some combination
- View Groups
- Create Group
- Create a smart group
- modify the criteria for a smart group
- Create a profile
- Edit a profile
- Click through reports looking for obvious errors
Batch Data Entry
- Create a batch data entry for contributions
- Edit a batch data entry for contributions
- Create a batch data entry for memberships
- Edit a batch data entry for membership
- Create a batch data entry for pledges
- Edit a batch data entry for pledges
Tests for the dedupe screen