Skip to end of metadata
Go to start of metadata

We plan on writing this integration in the following two parts so that it is as easy as possible for this to be extended to cater for other direct debit providers.

  1. An abstracted UK direct debit extension
  2. A smartdebit (http://www.smartdebit.com/website-direct-debits.html) specific extension

Requirements

Clients should add specific requirements here.

  • ability to integrate with auto renew membership functionality
  • report on cancelled / failed direct debits for follow up

Specification

Specification will be developed here.

Labels
  • None
  1. Jul 05, 2011

    I'm just trying to clarify terminology here. Does direct debit integration mean a) making an immediately processed transaction on a debit card that, for example, withdraws money from a chequing account and deposits it in the organization's bank account before the thank you page is displayed to the CiviCRM user? If so, then the abstracted part is more about the difference between the fields for a credit card versus the bank account number (in Canada that's the institution or bank number, the transit or branch number, and the account number itself), and the processing will follow the same general lines as the existing code that handles credit cards through payment processors. Or does direct debit integration mean what people here sometimes talk about as pre-authorized payments? In this, there is a direct debit made from the constituent's bank account that is credited to the organization's bank account. Many non-profits here make periodic, usually monthly, uploads of files containing a batch of debits to made of various amounts from various bank accounts. Perhaps you mean both ideas or something else again?

    In Canada the API interfaces for the first are very similar to those for credit card processing. The later are very different, requiring Header and Footer records for the file, and providing results in a downloadable file or two. As the transfers may take days to clear just like a paper cheque, the results aren't known instantaneously.

    (Apologies for not wading through the site to find out which is which - they seemed to be talking about the system for collecting and storing bank account information, which according to the Payment Card Industry Data Security Standard (https://www.pcisecuritystandards.org/security_standards/ ) is information that does not need to be encrypted, in contrast to Credit Card information.)

    Put another way, is there overlap between SmartDebit support and ACH (http://en.wikipedia.org/wiki/Automated_Clearing_House ) in the US, which implements the second approach that is also common amongst Canadian non-profits? I'm interested in collaborating on an ACH implementation.

    Finally, do you have financial goal based on the estimate of work yet?

    1. Jul 07, 2011

      My understanding is that it is closer to the first in that the customer fills in a form (possibly an iform or is redirected to the smart banking) to authorise their bank account to be direct debited on a regular basis ( not necessarily with a debit card). The api from what I saw last I looked was similar to a credit card processing API,

      Here is an example of an NZ direct debit processor page

      https://secure.flo2cash.co.nz/donations/oxfam/Donate.aspx?channel=gen

      1. Jul 07, 2011

        An excellent form.

        I think we should plan on the form of the banking information varying by region in terms of the number and length of fields. I've attached an API document used by the major Canadian payment processor which also details the requirements for the American system.

        The drop-down list of the names of banks is nice. I'm not sure how that would work in places where there are too many for a drop-down, or where the list of thousands changes to frequently to be feasibly tracked. It always seems to be included in paper forms.

        Other names for fields for direct debit / direct deposit that a UK firm (Packt Publishing) asked me for are: IBANN (Europe), SWIFT Code (not used in N. Am.), Bank Routing Code / Bank ID, Sort Code (UK).

  2. Jul 07, 2011

    I'm not familiar with the various systems in use around the world.  I'll try and clarify what happens with a UK direct debit.

    a direct debit MANDATE is the permission for an organistion to take money from a client bank account.  This normally happens on a regular basis.  It can be for a fixed or variable amount.  Setting up the mandate is seperate from actually making any payments.  i.e. take X fixed/variable amount from my account every Y days/months/years starting from date Z until further notice.

    To set up the mandate, the client needs to give the organisation their bank account number and sort code (branch code).

    Then the organisation (normally on a regular basis) asks their direct debit provider to process all the PAYMENTS.  Once this has been done, the direct debit provider reports on success / failure direct for each mandate.

    1. Jul 07, 2011

      This is exactly the process for authorizing direct debits in N. Am. The organization getting the money needs to track the authorization. In the US it seems to be common for the authorization to be for a certain number of payments. In Canada, it's almost always for an on-going regular amount with no end date.

  3. Jul 07, 2011

    We'd be very happy to come in on this and meet a proportion of the costs. It would be a  useful tool for us which we have been wanting to see for some time. The best example I have seen is the one Michael showed me  - https://www.concern.net/en/donate/regular/new?amount=

    It would be very helpful if the existing CiviCRM data on an individual were automatically filled in the form (name, address etc.).

    I hope I get a chance to meet you, Eileen, when you are here.

    Simon

    1. Jul 19, 2011

      Hello Simon,

      The best way to meet a proportion of the costs is on this page: http://civicrm.org/mih#ukdd.

      You'll need a credit card to make that payment.  If you don't have one, we can invoice you on behalf of CiviCRM.

  4. Jul 07, 2011

    Is it practical for the scope of this project, to make the architecture open to plugging in country-specific gateways? Such as being able to plugin a North American ACH gateway?  It will be easier to get donations to fund this project if there is a benefit (down the road) to US and Canadian nonprofts.

    1. Jul 08, 2011

      I'm starting work on a project right now that includes an ACH Gateway with Royal Bank of Canada and would like to collaborate with others on the general aspects of this. My sense is that 1) is better understood as an abstract direct debit plugin system, given the similarity between UK and N. Am. and NZ direct debit systems.

      1. Jul 08, 2011

        If you look at that NZ form it is hosted by the payment processor. I think Michael's form may have been too. In which case CiviCRM's interaction hopefully need not be too different to the way it interacts for credit cards

        1. Jul 09, 2011

          My client is interested in the second batch system for their 3 million members. This is a first step in moving away from a client server legacy system used only by central staff.

  5. Jul 12, 2011

    Has anyone investigated the approach for ACH direct debit taken by the module at:  http://drupal.org/sandbox/johnbarclay/1215340   This works with the APIs for a comany called "IPAY" (http://www.ipay.com/ )

  6. Jul 13, 2011

    Hi All
    We have been involved in Direct Debit for over 20 years with our flagship CRM products and have created a CiviCRM module which carried out the Direct Debit process. We'd be happy to walkthrough this module if needed and can contribute back if needed also.
    Let us know!
    Thanks
    Parvez
    NFPServices
    Hi All

    We have been involved in Direct Debit for over 20 years with our flagship CRM products and have created a CiviCRM module which carried out the Direct Debit process. We'd be happy to walkthrough this module if needed and can contribute back if needed also.

    Our module will produce the relevant lodgements file to send to the relevant BACS software.

    We have also done integration with a Direct Debit bureau where the direct debit mandates are held by the third party and not within CiviCRM.

    Let us know how we can help.

    Thanks

    Parvez

    NFPServices

    1. Jul 19, 2011

      Hi Parvez,

      Can you put the code in a public repository so that we can have a look?

      Thanks,
      Michael

  7. Aug 03, 2011

    Hello, 

    the requirements for UK DD are not the same as Debit Card (which is authorised prior to settlement). UK DDs are operated through BACS and they publish very specific scheme rules for electronic submitters. There are standardised messages for signing up new customers (called AUDDIS - Direct Debit Instruction), sending payment files (both Debit and Credit are possible under the scheme), receiving lists of failed Direct Debits (ARUDDs) and receiving lists of amendments and cancellations (ADACS). All of these messages are available electronically (definitions are XML) however most submitters either operate through a bureau (there are dozens to choose from) or licence their own BACSTEL-IP software from an approved supplier. If you go this route you have to go through a (reasonably thorough) accreditation process. 

    By the way if you sign up DD customers online (so called paperless DD) you need to pass separate accreditation, and your system needs to validate sort code and account numbers for validity. There are a number of third parties (Experian is one) that you can off-load this validation to; building your own database of valid sort codes and check digit rules is probably a step too far for an open-source project although I admire your ambition if you do attempt it. 

    DD capability will be a hugely useful add-on and I can offer some time and guidance to your project if you need it. I've implemented UK DD at several clients over the years.

    David

    1. Aug 03, 2011

      Thanks for the informed comment, David.

      Michael et al, just to advise you that we will be implementing direct debit batches to support one of our clients (partial spec at http://wiki.civicrm.org/confluence/display/CRM/Direct+Debit+and+Direct+Credit+-+Monthly+Batches). Their workflow currently involves paper forms that record the authorization from the donor, though I'm not familiar with the exact specifics.

  8. Mar 13, 2012

    Can anyone inform me on the status of ACH Transactions (one-off and batches) in the US for Civicrm?

  9. Jul 03, 2012

    One option for implementing this would be to use GoCardless (https://gocardless.com). We're a UK-based payment processing company, that uses Direct Debit. We have a simple RESTful API and an accompanying PHP library, which should make integration easy. Our pricing is extremely low - 1%, capped at £2. No setup fees, monthly fees, just 1%. The £2 cap means that everything over £200 is free, so it's certainly still viable for larger payments. Don't hesitate to get in touch if you've got any questions.

  10. Jul 06, 2012

    Hi Harry, Thanks for chipping in here.  I did chat with a developer of yours using your online chat a while ago.  We'd be really up for getting GoCardless integrated with CiviCRM as a solution for recurring payments.  There is a large potential CiviCRM customer base that would be interested in using a GoGardless integration if it were available.  Does GoCardless have an capacity to help write the integration? I would have thought that if you could get a CiviCRM payment integration expert and a GoCardless developer in the same room for a couple of days or so, we could write this integration.

    I'll send you an email as well.If would be great to talk further about how we can make this happen (michaelmcandrew@thirdsectordesign.org).

    Thanks,
    Michael

  11. Jul 09, 2012

    Hi Michael,

    We're certainly happy to help out with an integration. I'll send you an email with more details.

    Cheers,

    Harry

  12. Jul 11, 2012

    How does the BACS (http://en.wikipedia.org/wiki/BACS) system fit within this project? Is that system sitting behind the scenes of the API/Vendor that CiviCRM  connects to?

  13. Jul 19, 2012

    GoCardless (and other DD service providers tend to do the same, I think) take care of BACS for you.  i.e. you talk to GoGardless and they talk to BACS (possibly via some other intermediary - not sure about that) but the CiviCRM installation has no direct contact.

  14. Jul 19, 2012

    Just had a v. good meeting with Harry.  https://gocardless.com/docs/php/merchant_client_guide.  The API sits on top of the whole direct debit layer and keeps all the complexity hidden (mandates, payments) so we can just treat this like a recurring payment. i.e. set up the payment (offsite) and get notified each time the payment comes in (IPN). 

    There is also a pre-auth model so you can basically authorise the non profit to take variable amounts from your bank account which sounds like a good fundraising tool.

    There is no on site option so you have to go to the gocardless page and then get sent back.  There are some options for branding, and giving more branding options is on the road map (though harry can provide more details)  Also, it isn't currently possible to transfer existing direct debits, but it might be possible to address that at that at some point in the future.

    I think that the sprints are probably the best place to get these integrations written.  GoCardless might be able to send someone there in person - if not they have a campfire https://gocardless.campfirenow.com/3ae88 where they are around to help people write the integration.

    Michael

  15. Jul 30, 2012

    So I've been pushing this project forward in terms of the UK requirement.

    For the clients that I'm working with an API approach is required. I'll be talking to Smart Debit about the possibility of integrating with them tomorrow (Tuesday 31st July)

    If anyone has any other providers they wish to integrate with that have API's then please shout now. We'll be looking to get some support from them so would be good to get more names into the basket ASAP.

    Thanks

    Parvez

    1. Jul 30, 2012

      I spoke to Michael in quite a lot of detail a couple of weeks ago about integrating GoCardless (which has a really simple API for Direct Debit), and it seems like it'd be a really great fit. If you're pushing ahead with this, we'd be happy to help out with a GoCardless integration.

      1. Jul 30, 2012

        Hey Harry

        If you could email me on parvez at vedaconsulting dot co dot uk that would be great

        We can touch base tomorrow

        Thanks

  16. Nov 28, 2012

    Hi All

    We've completed Phase I of the UK Direct Debit integration and have a working demo at http://direct-debit.vedaconsulting.co.uk

    Please feel free to have a play!

    On the site we've also highlighted the areas we plan to work on for Phase II and Phase III. These will be turned into functional requirements soon.

    Thanks

    Parvez

  17. Mar 20, 2013

    I am trying to estimate the level of effort needed to update/convert the existing Drupal module payment processor for ACH ( via Vanco Services) to be a CiviCRM extension that can work with version 4.2 or better.

    The existing ACH/Vanco module is at: http://lamp1.axiaconnect.com/~filesbac/vanco/vanco.directpayment.processor_Civi4.1.x_D6.zip   thanks to BackOffice Thinking. It is also described at: http://civicrm.org/blogs/pkeogan/vanco-payment-processor-civicrm-33-34-and-41 ) 

    After digging into the code, I noticed that the module overrides a number of tpl files because the core tpl files only allow for credit card payments, while the module allows someone to pay either by credit card or ACH.  Could the changes to the tpl files be incorporated in the core tpl files? I was thinking there would need to be some kind of flag for each payment processor to indicate if it allows ACH or Credit Cards or both.  Then there is the situation where the payment processor in general allows both, but the organization using CiviCRM is only set up for ACH or Credit Cards with that processor. (eg they use Vanco for ACH only, and use Authorize.net for credit cards)

    (This has been already posted on the developer forum at: http://forum.civicrm.org/index.php/topic,28087.0.html

    How can I download the code for the UK Direct Debit integration, so that I can hopefully reuse some ideas/code.


  18. Jul 15, 2013

    hi,

    what's the current state of this, can someone confirm?

     

    Thanks in advance,

    Andy

  19. Jul 15, 2013

    Hi Andy

    We've been releasing beta versions of the code out for the last 3 months. The plan is to roll out the final code for the first week of August. Its now quite stable with a number of users. At the moment its Smart Debit integration only, GoCardless is in the process of being signed off and will follow a few weeks after the launch.

    Let me know if you need anything else.

    Thanks

    Parvez

  20. Jul 15, 2013

    thank you for such a very quick response!

    The client I'm working on is UK/EU, so I was looking at GoCardless for this, and Authorize.Net for card payments ... any advice you can offer? (smile)

     

    Regards, and thanks again for quick answer,

    Andy

  21. Jul 15, 2013

    Go Cardless should be fine, its a cheaper option and the donors/customers will see go cardless in their bank statements as the SUN is shared by all go cardless customers. If your fine with that then its the best approach. Not used Authorize.Net so cant offer any advice on that.

    1. Jul 15, 2013

      Harry from GoCardless here. This was the case until a few months ago. Our current strategy is to set up a mandate for each merchant a customer pays. This allows us to set a custom, merchant-specified reference on each mandate, which means that the correct name will appear on bank statements.

      Feel free to ask away if you've got any more GoCardless-related questions.

      1. Jul 15, 2013

        Hey Harry

        Thanks for the info. We'll be in touch in a week or so to discuss the completion of the integration.

  22. Jul 15, 2013

    Amazingly fast responses all round - good to see (smile) 

    Harry, may i ask what's the process a customer goes through to sign-up - UK incorporated entity (a not-for-profit according to Companies House rules, but not actually a registered charity), is it the same as if they are a commercial, or do you require extra info? I take it from what you said that the site owner / client can have their own name on DD transactions; are cross-EU transactions possible at this point? (IBANs needed?)

     

    Andy

    1. Jul 15, 2013

      Hi Andy, Kit here from GoCardless.

       

      If you are not a UK registered company, we would recommend signing up as an individual. There is no difference in terms of legal status of the account, or features on the account.

      The site owner / client will have their name on the bank statement alongside GoCardless.

      EU payments are still in development, but they are on the way and should be released later this year. Currently it is only possible to send and receive funds from a UK account.

      If you have any further questions, please do not hesitate to get in contact! You can also email us at help@gocardless.com or by calling 020 7183 8674.

       

      Hope everyone's enjoying the sunny weather,

      Kit

  23. Jul 16, 2013

    Hi Kit,
    Client is a UK registered company but is 'limited by guarantee' and a not for profit as their Companies House status/role, but not actually a registered charity. A fairly unusual position, perhaps.

    Will contact you directly to discuss further via the details you've provided, not appropriate to post more info on a forum?

    Sorry for slow response, was out enjoying the sunny weather at the time of your message (wink) 


    Andy

    1. Jul 16, 2013

      Hi Andy,

      Glad to hear you were enjoying the weather. The air-con in our office was broken yesterday, so I wasn't so much enjoying the weather, as melting in it.

      Sorry I misunderstood your first message. If the non-profit is registered with Companies House then they should be ok to sign up as a business (we grab our data directly from Companies House). If for some reason it doesn't work, then feel free to tell them to sign up as an individual, and to pop us an email at help@gocardless.com with the full company details, and we can just update it manually for them.

      Of course, feel free to contact us directly on the details provided (help@gocardless.com or 020 7183 8674), and you are more than welcome to ask to speak to myself directly. Our support hours are generally 9am-6pm mon-fri.

      Have a good one,

      Kit

  24. Jul 16, 2013

    thanks Kit,

    hope your air-con is working today - guessing you're in central London given phone number, am enjoying the sun in zone 2 but apparently it's going to get hotter this afternoon, so am definitely avoiding the tube today (smile)

     

    Will talk to client and be in touch directly,

    Andy

  25. Sep 18, 2014

    Any update on this spurt of activity last summer? Particularly with reference to GoCardless integration - I've a couple of clients that would be interesting in using goCardless if the integration was already there, although they don't currently have resources to fund any development.

    1. Sep 18, 2014

      Hey Graham

      Its almost done, if you could hang on post CiviCon we'll push out an update.

      1. Sep 18, 2014

        Hey Parvez - maybe we could briefly catch up on this at CiviCon. I know my client would prefer GoCardless to Smart Debit, so if it's very nearly in place I can probably convince them to wait a little longer...


Creative Commons License
Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution-Share Alike 3.0 United States Licence.