Skip to end of metadata
Go to start of metadata

(Suggested UI for Import and Dedupe contributed by Michal Mach)

 [brush added suggestion for informaion to be "added"/"appended" as well as overwritten or ignored.  This would depend on the type of field: eg., Location information could simply add a new location, while a custom text field could union the strings, while a radio button would not have this option.  I suggest that by default, but configurably, the maximal "append or add" option is default.]

Import could be a perfect Ajax application? I see at least two uses at the moment:

Interactive import log

Instead of progress bar, we could display import log as it is processed:
=====
Starting import process:

  • New contact John Doe added.
  • New contact Jane Doe added.
  • Possibly duplicate contact Allan Brown held for manual handling.
  • New contact Jane Brown added.
    =====
    .... etc - it could roll on the screen as the import goes. However, I'm
    not sure here - I think import goes pretty fast, so it might be overkill
    to transfer even short chunks of data as above back to browser every
    time new record is added during the import.

Searching for Duplicates

We'll also want to look for duplicates at other times than import.  The necessary information to "realize" that a contact is a dupe of another may only come later, eg. when we obtain an email address, phone number, etc.  The first need will be for a dialog that will allow a dynamic set of possible dupe-matching criteria.  This could be based on any subset of street addresses, phone numbers, emails, and names.  (Others)  Additionally, it will be valuable to include a certain amount of fuzziness in the matching to allow for typos, etc.  Perhaps as simple as how many characters different -- ie. if 1, 123 Main St would match 193 Main St but not 132 Main St.  Also would be good to include synonyms (St. and Street), and ideally some kind of "homonym-matching" like Google manages in its engine.  But that's luxury.

Duplicates handling could be interactive too

This is an ultimately useful gadget. It's difficult to explain, so I assembled quick
prototype to illustrate what I mean (see attached file). It would work like this:

1. At first you get a flat list of duplication suspects, there probably
should be also a column with action icons allowing to ignore, overwrite
existing record, or append fields (when possible). First Ajax use comes here: after clicking action
icon given row disappears.

2. If short status message doesn't give you enough hints to perform
quick action, you can click on the row (or one of it's elements, or
special "merge" action icon) and a comparison sheet is opened up for
you. It's also an Ajax widget - it gets loaded on demand.

3. In the comparison sheet you can merge the record manually. Clicking
"choose" or "add/append" has immediate effect, meaning that XMLHttpRequest is sent to
the server and given field is updated/created. This is easier in implementation
(as far as I know Ajax) and fully reversible until you click "Done
merging". Merged contact section in the comparison sheet is assembled by
default using merge settings that were defined before starting the
import process. When you click "Done merging", both given record and the
comparison sheet disappear thanks to Ajax magic.

4. You iterate over all the duplicate records until you resolved all the
conflicts. Remember, that during whole process you are keep staying on
the same page and all the communication with the server is happening in
small chunks through the Ajax mechanism.

Duplicate contacts: (I - ignore, O - overwrite, A - append or add)

Name

Surname

Data 1

Data 2

Duplicate with:

status

actions

John

Doe

married

republican

John Doe

identical

I O A

Allan

Doe

married

republican

Zoe Johnson

email identical

I O A

Dan

Brown

single

democrat

Dan Brown

New fields in imported
data

I O A

<
Detailed comparison:

Field

Imported contact

Existing contact

Merged contact

First Name

Dan

Dan

Dan

Last Name

Brown

Brown

Brown

 Location - phone

 123-333-4444 (choose) (add)

 123-555-6666 (choose)

 123-555-6666 AND 123-333-4444

Data 3

763527 (choose) (append)

543678 (choose)

763527 543678

Data 4 (radio)

no (choose)

(empty) (choose)

no

Data 5

(empty) (choose)

non-smoking (choose)

(empty)

 

 

 

Done Merging

John Doe married republican John Doe identical I O A

Labels
  • None
  1. Nov 30, 2006

    I like this proposed functionality a lot. As well as merging record etc in such a screen, it would be useful to be able to edit fields with additional information too, ie while fields with conflicting contents would be highlighted for attention (eg in red) it would also be possible to edit / update other fields as the same time, for example adding further details that are missing but may be obvious, such as country.

  2. Mar 02, 2007

    If we make the de-dup into a wizard the first step of the wizard is to define what fields to us in the duplicate matching. Basically the same interface as the import/export wizard field selection interface.

    I very much like the AJAX interface. Take a look at the new DIA user interface... might be a great starting point.


Creative Commons License
Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution-Share Alike 3.0 United States Licence.