08 Dec GCP – Integrate Salesforce into Google Cloud FHIR
How do I integrate patients from a Salesforce system into FHIR?
Introduction
Google Cloud Healthcare allows for standardized data exchange of healthcare data using the FHIR standard. FHIR stands for “Fast Healthcare Interoperability Resources” and is a standard that describes the data formats and elements for digital healthcare records, it also provides an API for the correct exchange of these records. This standardization greatly improves the way health records are stored and processed universally.
As a source, Salesforce was used. In this project we periodically sync all contacts in our Salesforce instance to our patients datastore in Google Cloud Healthcare. This sync happens via a Google Cloud Function that can be triggered to poll a certain number of unprocessed records from Salesforce, map them as described in the FHIR standard and then send them to Cloud Healthcare. New or modified records in salesforce should be synced with Cloud Healthcare. To know which records were already synced, a “datapointer” is saved in the Cloud Firestore. The datapointer simply is a record that saves the ID of the last processed record and its corresponding timestamp. This record gets overwritten at every successful sync between Salesforce and Cloud Healthcare.
Note that salesforce also supports event-driven integration patterns (instead of polling) using their internal CDC and platform events. More on this here. Additionally, GCP is releasing Apigee Integration which promises to further simplify these types of integrations.
This hands-on blog consists of following parts:
- Salesforce setup
- Cloud Functions Setup
- Cloud HealthCare Setup
Prerequisites
- A Google Cloud account with a billing account enabled
- A Salesforce Developer Account
- A Salesforce instance
- Postman for API testing
Salesforce
The setup for Salesforce is quite simple. As you can see on the picture below, we have a working account on an active Salesforce instance. We’ve already added some Contacts for later testing.
Next, we need to enable our Salesforce instance to be accessible to external systems. This is done via “Connected Apps”. You can find the official documentation for connected apps here.
From the gear-icon in the top-right, navigate to Setup. In the search box on the left, type “App Manager” or find it in the side menu. There you will find a couple of pre-built connected apps. We’ll add our own by selecting “New Connected App” in the top right.
Fill in the fields as shown on the image; connected app name, API name and Callback URL can be filled in as you wish. In the callback url field, you can also just fill in http://localhost/callback . Make sure your email address is the one used to create your salesforce developer account and check if the correct OAuth scope is selected. Save the connected app, Salesforce takes several minutes to finalize the setup. Once it is finished, you’ll be able to reveal your Consumer Key and Consumer Secret, we’ll need this later.
The only thing we need now is a security token. Go to profile settings, select “Reset my security token” and follow the instructions to get the token.
Test the connection using Postman
Here I will show how to make testing to your Salesforce instance API a bit simpler. When sending a request, you will need some kind of authentication. In our case this will be via the accesstoken. The only annoying part is that this token expires quite fast, you then have to request a new one, copy it and paste the new one in the request you wanted to test out. We make this a lot simpler with Postman environments. Create a new one and fill in the fields as shown in the picture below.
Normally, you only need to fill in the fields on the left (“Initial Value”), and it should fill in the “Current Value” column on the right.
Client-id and client-secret are the “consumer key” and “consumer secret” that can be found on you connected app, as discussed earlier. Username is just the email you used to create your Salesforce developer account. Leave “instance-url” and “access-token” empty, as these will be filled in automatically after the request.
Next up is the POST request to get our accesstoken using OAuth2. Make sure that you selected POST, checked “x-www-form-urlencoded” and chose the previously created environment in the top right corner so that the request has access to the variables. Also, note that the password in the request consists of the “security-token” appended to the “password” variable.
In the response we get what we need to send other requests to our Salesforce instance. If you go and look back at the environment we created, we should see that “instance-url” and “access-token” are now filled in.
Request a contact from Salesforce
If you select a contact on Salesforce, you can find their specific ID in the URL.
Now create a simple GET request in postman using the url:
- {{instance-url}}/services/data/v{{version}}/sobjects/Contact/{ID}
With ID replaced by the ID of a contact. In the headers section add:
- Autorization Bearer {{access-token}}
If you don’t have the other headers as shown in the image, don’t worry, you don’t need to fill these in. Make sure you’re still using the salesforce environment and press Send.
If this works, your salesforce setup was successful and we can now request data from it.
You can request specific fields or records by using SOQL (Salesforce Object Query Language). You can find the documentation here. We will use this in the Cloud Function code later, but you can already play around with this example GET request:
- {{instance-url}}/services/data/v{{version}}/query/?q=SELECT Id, Name FROM Contact
You can find the fieldnames of the contacts under setup > ObjectManager > Contact > Fields and Relationships.
If the request fails with a “token expired” or “access denied” error, run the “Request OAuth Token” from the previous step again, it will update the accesstoken.
Cloud Functions Setup
For this step you’ll need a working Google Cloud Account (there’s also a free trial), with billing enabled.
As this setup is quite generic, I’ll refer to the docs. Node.js version 14 was used for this project (more on this later).
We used HTTP as a trigger for the function.
After this, learning how to deploy local code to Cloud Functions will be essential and can be done very easily using the gcloud sdk. Here are the docs. We will do this with the actual code in a later step.
Cloud Healthcare setup
On Google Cloud Platform: go to “Healthcare” in the side menu and enable the API.
Create a dataset and choose the location closest to you.
In the dataset, create a datastore Type: FHIR Choose version STU3 and check “allow update create”, leave the rest checked. Skip the rest
Follow the steps here to be able to access the Cloud Healthcare API (FHIR) from your code. There are multiple ways to authenticate, we went for the “finding your credentials automatically” way using powershell. Make sure to use a relative path when referencing the authentication json file. It’s also smart, when working in a git repository, to add this json file in .gitignore if you want to keep this file within the project. Just make sure the credentials don’t end up online.
Setting up Firestore for our datapointer
On Google Cloud, go to Firestore in the sidemenu. Create a collection, then a document within that collection, and then the fields resourceId (as a string) and lastUpdatedAt (as a timestamp). It is a good idea to already set this timestamp way back in time, so that we’re sure all records since then are processed from Salesforce.
The code
The final step in this setup is the actual code for the Cloud Function. You can find the source on our GitHub and recreate it using the README provided there: https://github.com/I8C/gcp-sfdc-to-fhir
We used typescript to have a type safe language and terraform to simplify the creation of the resources. Both are not necessary of course, and are described in the readme if you wish to use them.
To explain the code, we will divide it into steps to simplify things.
“pollPatientBatches” is the name of the function that we will deploy to Cloud Functions.
We need quite a bit of environment variables to connect to the several instances we created. The names speak for themselves, so fill in accordingly. Create a .env file like this one and don’t forget to add it to your .gitignore file.
We also need to add these to our Google Cloud Function under “variables”. The easiest way is just to do this on Google Cloud manually, although you can also do it via the gcloud SDK.
Part 1
First, we initialize some connections with our instances and we get the current datapointer in firestore.
Part 2
Here, we use the “jsforce” npm package to send a request to Salesforce with a SOQL query, asking for the fields we want to sync with Cloud Healthcare. Notice that we order by lastModifiedDate and limit by 10 records to ensure that when setting the datapointer, it is always set to the last of the 10 records. We fetch these records and push them into the fetchedRecords array. We only request 10 records at a time to prevent overload, but this number can of course be changed according to the need.
Part 3
This is the part where we get ready to connect to Healthcare, so we authenticate and set some environment variables. We also set our “lastProcessed” variable to the timestamp and ID of the last updated record of the 10 we received back from Salesforce. Later we will update the timestamp in Firestore with this variable.
Part 4
Next, we loop over every record, map it to the correct structure for FHIR and update it. Note that if enableUpdateCreate is set to true in Healthcare we are able to update existing records, but also create them if the id was not found (i.e., the record was not yet created within Healthcare). This feature saves a lot of data transfer.
Part 5
At last, the only thing we need to do is to update the datapointer to our lastProcessed variable from part 3.
Cloud Scheduler
At last, you can opt to trigger this function periodically using the cloud scheduler. Simply go to Cloud Scheduler and click “Create Job”. Give the schedule a name, a frequency (e.g. “* * * * *” for every minute, more info here) and a timezone. Then target type: HTTP, URL: the url of your cloud function and http method: GET.
Conclusion
At the end you should be able to trigger the function and it should sync contacts in Salesforce with Google Cloud Healthcare, updating the datapointer in Firestore accordingly.
Author: Damien Mesure 
