E-mail
There is no requirement to change the UI of CiviCRM for this project. The same user behaviour that affects financial data in CiviCRM should be possible (perhaps with some small exceptions) at the start and end of the project. The difference will be in the data in the database.
Regression testing on browser UI
Selenium integration (functional) tests for all CRUD operations will be created at the beginning of the project to verify the user interface operations for both records and fields. They will test that browser actions result in browser results. The read operations will include searches, dashboards, and existing reports. Many of these tests already exist, and the core team will take responsibility for defining the rest. These user level tests will constitute one class of regression tests that will ensure existing functionality does not break as the data schema and associated code are refactored.
Integration testing of browser actions and DB changes
Additional functional tests based on the input behaviour parts of the regression tests will verify the database representation of these records and field on the 4.0 schema. The first part of the detailed work specification will be the development of 4.1 versions of these database tests that check for the correct representation of the data in the new 4.1 schema.
Integration testing of upgrade actions and DB changes
A separate suite of functional tests will validate data migration. While design and development are still to come, at a high level it seems possible to store a copy of a 4.0 database, run the 4.1 upgrade against the original, and then run tests comparing the schema structure, records, and fields in the copy of the 4.0 database and the 4.1 database. These integration-level comparisons will reflect the differences in the database tests above for 4.0 and 4.1 operations. Ideally there should be reuse of the data validation operations between the integration testing and the migration integration testing.
Regression testing of API actions and DB changes
A specification for changes to the API still needs to be scoped and developed. I am unclear whether it will be possible to retain backward compatibility of the 4.1 API with the 4.0 API. That is, it may be possible in some cases but not all to map a call against a 4.0 BAO object to one or more calls against 4.1 objects. To the extent that it is possible and funded, revised 4.1 versions will be developed of 4.0 integration tests of API calls that verify database record and field values before and after the call and returned values.
For contracting, more traditional specs and procedural definitions of work will be developed.
For spec, there needs to be detailed description of any change in the semantics of a field / record.
