(Suggested UI for Import and Dedupe contributed by Michal Mach)
[brush added suggestion for informaion to be "added"/"appended" as well as overwritten or ignored. This would depend on the type of field: eg., Location information could simply add a new location, while a custom text field could union the strings, while a radio button would not have this option. I suggest that by default, but configurably, the maximal "append or add" option is default.]
Import could be a perfect Ajax application? I see at least two uses at the moment:
Interactive import log
Instead of progress bar, we could display import log as it is processed:
=====
Starting import process:
- New contact John Doe added.
- New contact Jane Doe added.
- Possibly duplicate contact Allan Brown held for manual handling.
- New contact Jane Brown added.
=====
.... etc - it could roll on the screen as the import goes. However, I'm
not sure here - I think import goes pretty fast, so it might be overkill
to transfer even short chunks of data as above back to browser every
time new record is added during the import.
Searching for Duplicates
We'll also want to look for duplicates at other times than import. The necessary information to "realize" that a contact is a dupe of another may only come later, eg. when we obtain an email address, phone number, etc. The first need will be for a dialog that will allow a dynamic set of possible dupe-matching criteria. This could be based on any subset of street addresses, phone numbers, emails, and names. (Others) Additionally, it will be valuable to include a certain amount of fuzziness in the matching to allow for typos, etc. Perhaps as simple as how many characters different -- ie. if 1, 123 Main St would match 193 Main St but not 132 Main St. Also would be good to include synonyms (St. and Street), and ideally some kind of "homonym-matching" like Google manages in its engine. But that's luxury.
Duplicates handling could be interactive too
This is an ultimately useful gadget. It's difficult to explain, so I assembled quick
prototype to illustrate what I mean (see attached file). It would work like this:
1. At first you get a flat list of duplication suspects, there probably
should be also a column with action icons allowing to ignore, overwrite
existing record, or append fields (when possible). First Ajax use comes here: after clicking action
icon given row disappears.
2. If short status message doesn't give you enough hints to perform
quick action, you can click on the row (or one of it's elements, or
special "merge" action icon) and a comparison sheet is opened up for
you. It's also an Ajax widget - it gets loaded on demand.
3. In the comparison sheet you can merge the record manually. Clicking
"choose" or "add/append" has immediate effect, meaning that XMLHttpRequest is sent to
the server and given field is updated/created. This is easier in implementation
(as far as I know Ajax) and fully reversible until you click "Done
merging". Merged contact section in the comparison sheet is assembled by
default using merge settings that were defined before starting the
import process. When you click "Done merging", both given record and the
comparison sheet disappear thanks to Ajax magic.
4. You iterate over all the duplicate records until you resolved all the
conflicts. Remember, that during whole process you are keep staying on
the same page and all the communication with the server is happening in
small chunks through the Ajax mechanism.
Duplicate contacts: (I - ignore, O - overwrite, A - append or add)
Name |
Surname |
Data 1 |
Data 2 |
Duplicate with: |
status |
actions |
|---|---|---|---|---|---|---|
John |
Doe |
married |
republican |
John Doe |
identical |
I O A |
Allan |
Doe |
married |
republican |
Zoe Johnson |
email identical |
I O A |
Dan |
Brown |
single |
democrat |
Dan Brown |
New fields in imported |
I O A |
<
Detailed comparison:
Field |
Imported contact |
Existing contact |
Merged contact |
|---|---|---|---|
First Name |
Dan |
Dan |
Dan |
Last Name |
Brown |
Brown |
Brown |
Location - phone |
123-333-4444 (choose) (add) |
123-555-6666 (choose) |
123-555-6666 AND 123-333-4444 |
Data 3 |
763527 (choose) (append) |
543678 (choose) |
763527 543678 |
Data 4 (radio) |
no (choose) |
(empty) (choose) |
no |
Data 5 |
(empty) (choose) |
non-smoking (choose) |
(empty) |
|
|
|
Done Merging |
John Doe married republican John Doe identical I O A |

2 Comments
Hide/Show CommentsNov 30, 2006
Peter Hirst
I like this proposed functionality a lot. As well as merging record etc in such a screen, it would be useful to be able to edit fields with additional information too, ie while fields with conflicting contents would be highlighted for attention (eg in red) it would also be possible to edit / update other fields as the same time, for example adding further details that are missing but may be obvious, such as country.
Mar 02, 2007
David Geilhufe
If we make the de-dup into a wizard the first step of the wizard is to define what fields to us in the duplicate matching. Basically the same interface as the import/export wizard field selection interface.
I very much like the AJAX interface. Take a look at the new DIA user interface... might be a great starting point.