On-Page Address Standardization for Canada Post

Currently, CiviCRM supports the USPS service for address standardization.

I would like to create a service to provide a similar service for Canadian postal code data. The intent is to ensure that Canadian addresses are valid and have been standardized at time of entry, before save is allowed. (Perhaps there could be some configurable option to allow save after a warning message has been displayed. Not positive about this use case.)

The project will have two components. One will involve implementing the provider using the Canada Post Address Data product. The Tech Spec for this product and some sample data are attached to this page (see icon on upper left of page).

This service will provide an interface modeled as much as possible on the USPS one currently supported by CiviCRM. (Detail tbd). The feature will be controlled in the same way that the USPS integration is (Administer | Configure Global Settings | Address Settings), though will only require the Web Service login and password to use.

The second will be to code a consumer to operate within CiviCRM. This will involve a) creating a general PHP class similar to what is available for USPS that is not specific to CiviCRM, and b) creating CiviCRM code to make use of this class in appropriate ways within CiviCRM.

We will use the CiviEngage address parsing code to allow for better matches. The basic functionality will work like a civicrm_hook_validate function that validates the address when a contribution page is saved. As the current USPs code does not require the address to validate in order to save, we will need to allow admins to decide if they want to force validation or not (the data on the Web Service could be old or have errors in it, I've seen both of these scenarios dealing with CASS certification software in the US). We may want to look at setting up "fuzzy" logic that allows partial matches if we are going to force validation ( in order to allow for addresses that are built in between data updates to the web service).

The service will provide up to x lookups for free per day per requesting IP, and up to y thousand lookups per day at a cost of $z per year subscription. (Pricing subject to change.) These throttle limits will be used to comply with data license restrictions against scraping and bulk download.

The intent is to provide Canadian address verfication services for CiviCRM installations initially.

A second bulk interface will allow all Canadian addresses in a CiviCRM installation to be verified / standardized. The intent is to support address standardization / verification for new subscribers, and also after bulk address import operations.

The hit to the web service will be authenticated with a username / password that is issued by a Drupal site. People will be able to purchase access levels for their account through a manual process for now (later on a public facing site), and free accounts will be capped at appropriate levels of service. Though initially there will be no purchase necessary while it gets up and running.

Draft Specification for CiviCRM Integration

Project Estimate

Timing: Estimated delivery of code for acceptance testing is 4 weeks after signed contract, and availability of web service (which ever comes later).

Address Lookup

• submit address data to web service
• web service returns service status (e.g. valid request, unknown subscriber, overlimit etc.), address status (e.g. address ok, address corrected, address can not be processed / incomplete / unrecognized). For address corrected status, web service also returns the correct address

User address input

Configuration

• Public mode (profile / event registration / contribution page input): "no standardization", "force standardization", "optional standardization". Optional means user can over-ride suggested format and save an unrecognized address.
• Back-office mode (add / edit contact form): "no standardization", "force standardization", "optional standardization". Optional means user can over-ride suggested format and save an unrecognized address.

NOTE: Validation is only performed if Country field value is set and value = Canada AND at least a Postal Code value is present. NOTE: Setting default country to Canada for the site ensures that the country is set on form load. We do NOT save addresses if the only field with a value is Country and the country value = default country.

Inline (ajax) validation

1. Do a call to lookup service in the following cases:
   1. when leaving address block (submit all values). This applies to back-office contact create / edit form only since profiles do not define an 'address block or fieldset'.
   2. when leaving a new / changed value in Postal Code field (submit just postal code).
   3. when leaving a new / changed value in Street Address or City, so long as the other is non-blank (submit just street address and city).
2. If the lookup succeeds and returns a unique value for blank fields in address, then fill them in (eg submitting just a postal code will return most if not all other fields). NOTE: Street addresses can not be inferred from postal code AFAIK. dgg
3. If the lookup succeeds and returns non-matching values for non-blank fields, put an error message into form for each problematic field. Error message should include the correct value for the field as returned by the lookup service.
4. If the lookup returns an error (eg invalid postal code) then put an error message into form as follows for the respective kind of call in 1 above:
   1. Address block (back-end contact create / edit only).
   2. Postal Code.
   3. Both Street Address and City fields.
5. On form submit, run complete address validation and return error messages if appropriate (see next section).

Validation behaviors on form submit

• If "force standardization" - recognized addresses are automatically saved with recommended format. Unrecognized addresses throw a form-rule error. User must correct to save the address.
• If "optional standardization" - unrecognized address gives a warning. User can choose to modify or over-ride and save their input.

Address validation in Profile forms

Optional or required address validation will be performed on Profile forms (standalone or embedded in contribution / event registration pages) IF the Country value = Canada AND following fields are present in the profile:

• street address
• city
• province
• postal code
• country

Batch standardization

• Command-line script passes in CiviCRM / Drupal authentication. Retrieves subscriber info from database.
• Option to standardize all addresses, or limit to a single group of contacts
• Service status errors and specific unrecognized address statuses (with link to contact record) are written to a log

Address validation hook

Add a hook which allows developers to specify additional behaviors based on results of address validation call (form submission validation).

Example use case for the hook: Developer wants to set a value in a custom field based on the results of the address validation call.