CommCare
App Overview
CommCare is a powerful data collection platform developed by Dimagi. It is an open-source platform, and is primarily best for mobile case management.
Data Model
CommCare data is primarily stored in forms and cases. Forms are the building blocks of applications and cases are used to track data on objects, usually people. Learn more about CommCare forms and cases at the links below.
- Case management:
https://confluence.dimagi.com/display/commcarepublic/CommCare+Fundamentals+-+Case+Management - Form and case data in CommCare: https://confluence.dimagi.com/display/commcarepublic/CommCare+Fundamentals+-+Data+in+CommCare
Integration Use Cases
Example user stories:
- As a community health worker I want to store patient data that was collected in rural areas with no internet access in Salesforce so I can better analyze the data being collected and make more informed recommendations.
- As a teacher I want to set up automatic message alerts to my students in order to increase participation.
APIs & Integration Options
CommCare offers a number of integration options for extracting and/or loading data to and from CommCare HQ.
Web API
CommCare has different APIs for reading vs. updating data. Some helpful links:
Webhook: Forward cases and/or forms from CommCare to OpenFn using REST service
See CommCare docs on how to configure this webhook to "push" data to an external system like OpenFn. This option is great for real-time data forwarding.
In order to connect CommCare with OpenFn v2, you'll often need to set up CommCare data forwarding for individual forms, and for specific case types. Let's set up a connection to OpenFn and then see how to do each.
Creating a Connection
- Go to "Project Settings".
- Click "Connection Settings".
- Choose "Add Connection Settings" at the bottom
- Give the connection a name, and indicate email address(es) to send failure notifications to. See the CommCare docs for more on this..
- Paste the URL of the OpenFn webhook you want to forward data to
- If you have [webhook authentication]https://docs.openfn.org/documentation/webhook-security) set up on OpenFn, add the authentication type, the username and password here
- You can test the connection, then save it
Forwarding Individual Forms
- Click over to "Data Forwarding".
- Under "Forward Forms", click "+ Add a service to forward to"
- Select the connection to forward the forms to set up following the steps above
- Name it
- Select "POST" HTTP Request Method
- Choose "JSON" as Payload Format
- Exclude any (eg. test) users - forms submitted by them won't be forwarded
- "XMLNSes of forms to include" lets you select which form(s) to forward by adding the XMLNSes of the required forms. Follow this CommCare guide to find the XMLNS of any form. To add multiple, separate them with commas, spaces or newlines. Leave empty to forward all forms.
- Hit "Start Forwarding" to save and activate
Forwarding Specific Case Types
- Go to "Data Forwarding".
- lick "+ Add a service to forward to" under "Forward Cases"
- Select the connection to forward the cases to
- Name the forwarder
- Select "POST" HTTP Request Method
- Choose "JSON" as Payload Format
- Select which case type(s) you want to forward, for example "patient"
- Exclude any (eg. test) users
Data Forwarding and OpenFn Workflow Design
A clean way and efficient way of designing CommCare webhook workflows on OpenFn v2 is to have a separate workflow handling each form and case type. As each of your OpenFn webhook workflows has a unique URL, you'll need to set up a separate CommCare connection for each, then use that connection to forward the relevant form or case type to OpenFn. So your CommCare Data Forwarding overview might look like this:
While on OpenFn, you can set up one or multiple jobs to handle the form or case type received.
Pulling Data From CommCare
You can also fetch data from CommCare initiated from an OpenFn workflow, by sending a GET request to one of CommCare's data APIs. You could use our http adaptor to achieve this. Fetching can be done on a scheduled basis, as a daily or monthly sync. This design also lends itself well to processing data in bulk.
App Setup & Integration Tips
App installation and configuration
- CommCare docs on installing the mobile app: https://confluence.dimagi.com/display/commcarepublic/Install+CommCare+for+Android+Smartphones
App Versioning
Ensure that you are always using the latest app version when testing, by updating your app and checking that the version matches the latest version in CommCare HQ.
Exporting Metadata
Use this
link
to export form contents. This will give you access to all the question ids
and labels
in the CommCare form that are helpful for designing your
integration and mapping data elements.
Unique Identifiers
As CommCare data is stored in forms and cases, there are two types of UIDs in CommCare:
case_id
&form id
. You can search for a particular case or form submission in CommCare by using theFind Data by ID
feature here: https://confluence.dimagi.com/display/commcarepublic/Find+Data+by+ID. For example, if you received a submission in the OpenFn inbox that you would like to find in CommCare, search byform id
. If you'd like to find a particular case (i.e. person, event, etc) search bycase id
.
In the OpenFn message:
id
is the unique identifier for the form submissioncase_id
is the unique identifier for the case being updated by the form. For example, thecase_id
can be the UID for a person.
N.B. Unique identifiers and hidden fields
in CommCare forms: Make sure
that unique identifiers for forms and objects are always in the form. If the
unique identifier isn't relevant for the user, it can be added to the form as a
hidden field
. Learn more about hidden fields
here:
https://confluence.dimagi.com/display/commcarepublic/Hidden+Value+Calculations+Tutorial
Data Element Mapping Notes
Labels
are generally not mapped in an integration because they represent text
that is displayed to the CommCare user to facilitate easy use of the
application. Ids
however, represent actual data that should be mapped.
- When mapping
multiple choice
questions, make sure to consider how the answer choices should map to the source/destination system.
CommCare Credentials
To connect to CommCare you'll need a username, password, host URL, and the "appId".
The username is your full email address and the "appId" is the UUID which
designates your CommCare project as different from everyone elses. It can be
found in the URL of your application when you first enter it from the project
screen. I.e., the last part of this URL:
https://www.commcarehq.org/a/YOUR_PROJECT/apps/view/YOUR_APP_ID/
The raw JSON of your credential (for use in the CLI or when inspecting
state.configuration
) is defined in the
CommCare Configuration
section.
Common Errors
Docs in progress!
OpenFn Adaptors
OpenFn implementations can leverage both the
HTTP
and
CommCare
adaptors to connect
with the CommCare API.
Implementation Examples
- MiracleFeet (CommCare-to-Salesforce sync): https://github.com/OpenFn/miracle-feet
- Lwala (CommCare-Salesforce 2-way sync): https://github.com/OpenFn/lwala
- Grassroot Soccer (CommCare-to-Salesforce sync): https://github.com/OpenFn/grassroot-soccer