Aller directement à la fin des métadonnées
Aller au début des métadonnées

Go back to the Amnesty International Spain's Projects.

Project Overview


This project specifications have been written down into an attached document: Civi Data Integration v1.2.pdf.

  • Aucun
  1. Aug 25, 2014

    JoeMurray dit :

    What is your timeframe for thie parts of this.

  2. Aug 26, 2014

    The developments are going to take place during September. We hope we'll have a first v1.2-beta for the CiviCON London. Any idea or suggestion is welcome! (sourire)

  3. Aug 26, 2014

    There is still a fifth enhacement I'd like to include in v1.2:

    • Show field descriptions (when working with filters and parameters) instead of showing field names.

    An example on how to obtain field descriptions is available at civicrm/api/doc.

  4. Aug 27, 2014

    JoeMurray dit :

    Nice bite-sized agile approach. Currently I'm considering whether to try to use Pentaho Data Integration on an upcoming data migration project. I've heard that it can be quite time-consuming to code all of the transformations and mappings. What's your sense of the pros and cons of using it for a one-way data migration for a system with contacts, memberships, contribution payments for the memberships, events and event registrations, and a few custom activity types, all with lots of custom fields? The alternative would be to code scripts in PHP and SQL.

  5. Aug 28, 2014

    At the beginning, to create a transformation takes a lot of time. The logic of Pentaho Data Integration is very different than the logic of a PHP script. Anyway, once you have some experience, it becomes a really fast development tool.

    The main reason why I would recommend Pentaho Data Integration for a one-way migration is that the processes are really easy to modify. If you forgot to include a field, it would probably take only a few clicks to add it. If you were working with a PHP script, even if your code was clean, to make changes is a much slower process. My point is that a one-way data migration is normally a big and complex process, so you'll start with a draft (in Pentaho Data Integration, or in a PHP file) and you'll refine it in a series of iterations. This way of work, based on iterations, uses to be much faster using Data Integration tools.

  6. Aug 29, 2014

    Maybe a sixth enhancement to include in v1.2:

    • I've been making tests of the CiviCRM output with a big number of records and it looks like it makes the petitions in a one-to-one basis. It would be interesting to see if it's possible to create threads for every record, so a record doesn't has to wait for the previous one to be done.

    Next Monday I'll try to close this document, so I'll consider including the things we've been commenting here.

  7. Aug 29, 2014

    New improvement I'll consider including:

    • If a parameter with the name id is being passed, and the value corresponding to it is null, the parameter won't be included in the API call.

    That will facilitate upsert (update or insert) operations.

  8. Sep 03, 2014

    After a conversation with Juan (StrateBi), we've decided to change the approach a little bit. The idea is, instead of creating a new type of step that exposes the possible actions, we'll modify both the input and output steps. The input step will allow us, basically, to use any kind of get action. The output step, will allow us to use the actions create and delete. I'll update the document as soon as possible.

Creative Commons License
Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution-Share Alike 3.0 United States Licence.