Data integration is a core component of every company’s data management practices – whether it’s moving data to a data warehouse or migrating data from one or multiple databases to another. Fivetran makes it easy to extract, load, and transform your own data, but when it comes to your customers’ data you suddenly have to manage credentials. Handling and managing credentials introduces security issues, not to mention the hassle of updating credentials when necessary can be cumbersome.

Customers don’t want to share sensitive credentials to production applications and databases if they can avoid it. Additionally, you don’t want to be liable for potentially mishandling them.

So, what’s a solution to this? Sending them a link to a Connect Card.

If you’re a Powered by Fivetran customer, you may already be familiar with the concept of Connect Cards. Connect Cards are a Fivetran setup form that can be embedded in the UI of your application or portal. This makes it so your customers can authenticate a connection between their sources and Fivetran, entirely on their own.

But what if you don’t have an application or portal, don’t want to add a customer to your account, or are early in your development journey embedding the Fivetran API and need data ASAP? That’s where generating and sending out a link to a Connect Card comes in handy. In this guide, I share 6 steps to help you successfully get started with building a solution to this pain point, and by the end, you’ll know exactly how to solve this problem programmatically using Python.

Step 1: Create a destination

The first step in creating a link to a Connect Card is to have a destination set up on your Fivetran account. If you haven't already, set up Snowflake as a destination (or any of our other supported destinations). Once you're done getting it set up, grab the destination group ID from the connection details tab as shown in the image below. 

Step 2: Gather authentication credentials

After you have your destination group ID, you’ll need to gather your accounts API_KEY and API_SECRET so that we can make requests against the Fivetran API. These two can be found in the settings section of your Fivetran account as shown in the image below.

Step 3: Build your Python file

Once you have your credentials, we can start building a Python script to automatically generate links to connect cards whenever you need them. Here’s a link to the repo we’ll ultimately be building. Check it out to follow along.

We’ll start with importing a few Python libraries to make our lives easier, set up some environmental variables so that we don’t expose our credentials in code, and structure our auth headers appropriately.

#!/usr/bin/env python3
 
import json
import requests
from requests.auth import HTTPBasicAuth
import environ
 
#Reads the API_KEY and API_SECRET from .env file.
#Create one in the same directory as this file.
env = environ.Env()
environ.Env.read_env()
API_KEY = env("API_KEY")
API_SECRET = env("API_SECRET")
 
auth = HTTPBasicAuth(API_KEY, API_SECRET)
 
headers = {
   'Authorization': 'Basic ' + API_KEY,
   'Content-Type': 'application/json'
}

Now that we’re set up to make requests against Fivetran’s API, I’m going to walk you through each API request required to accomplish the end result – a link to a specific Connect Card.

In the first API request, we will make a request against Fivetran’s API and parse the response body to collect the connector_id. The connector_id is used to identify the unique connector in the overall Fivetran account which can have many different connectors running in parallel.

In the second API request, we will do the same thing as the first step, but against a different endpoint in order to collect the token. The token is tacked onto a predefined URL scheme that prompts the Connect Card experience when an end-user enters the link in their web browser.

In order to make these requests to create the connection between the connector and Fivetran, we will need to create what’s known as a payload to communicate with Fivetran’s API server. Once the server validates the payload we send, it will communicate back response data that we talked about in the previous paragraph – our connector_id and token.

Let’s dive into creating the payload programmatically.

Step 3: Create a payload

In order to create the payload, we need to define the service, group_id, and config in JSON format. To find your connector’s service name, head over to our documentation center and click on the connector name in the cell table that contains the compatible connector types. Next, find the first field entitled “service” — this is our service ID. You’ll see the service defined in our payload via a key/value pair.

Next, we will put in the group_id that we gathered at the very beginning of this guide which will place the connector inside that destination group.

The final step is to name the schema in the config section of our JSON payload. It’s important to name the schema because when the information lands in the target destination, it needs to have a unique identifier to identify it.

Once we have all of this information, your payload should look like the object below.

payload = {
   "service": "adwords",
   "group_id": "", #put in your group_id
   "config": {
       "schema": "cerebro_test"
   }
}

Now that we have this payload defined, we’re going to use it to request our connector_id and token from the Fivetran API.

Step 4: Generate the connector ID

To generate the connector’s ID, we’re going to create a POST request against the following endpoint:

POST api.fivetran.com/v1/connectors

Take a look at the Python code sample below that walks through the POST request and response parsing. In the code below, you’ll notice that there’s a `json=payload` parameter. It’s in this part of the program that we are passing the payload into our request that gets sent to the server.

#Create the shell connector
response = requests.post(base_url + 'connectors', auth=auth, json=payload).json()
 
#Parse out the the connector_id
connector_id = response['data']['id']

After making the request to the server, we need to parse through its response to us, specifically to find the connector_id that was generated. We store the connector_id sent to us as a variable and pass it into the next POST request, which will create the unique token for the Connect Card.

Step 5: Set up the token

To generate the token, we’re going to initiate a POST request against the following endpoint:

POST api.fivetran.com/v1/connectors/<connector_id>/connect-card-token

Take a look at the Python code sample below that walks through the POST request and response parsing. You’ll notice that we are appending the connector_id onto the URL in order to generate this request.

#Create the connect card token
response_token = requests.post(base_url + 'connectors/' + connector_id +
   '/' + 'connect-card-token', auth=auth, json=payload).json()
 
#Parse out the connectors token
token = response_token['token']

Similar to Step 4, we need to parse through Fivetran's API response to find the token that was generated. We store the token sent to us as a variable and use it to generate the link to the Connect Card.

Step 6: Generate the link

Now that we have the token generated, we need to append it to the URL that Fivetran provides per our documentation. After it is appended to the URL, we need to execute our final step of generating the link. To accomplish this we’re going to print out the link to the terminal so you can copy and paste it into any communication medium that you’d like.

#Generate the connect card link
print('https://fivetran.com/connect-card/setup?redirect_uri=fivetran.com&auth=' + token)

Your finished URL will look something like this below:

https://fivetran.com/connect-card/setup?redirect_uri=fivetran.com&auth=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJkIjp7ImxvZ2luIjp0cnVlLCJ2ZXJpZmljYXRpb24iOmZhbHNlLCJyZXNldCI6ZmFsc2UsImpvaW4iOmZhbHNlLCJiYWNrc3RhZ2UiOmZhbHNlLCJ1c2VyIjoibGl0ZXJfaW50ZWd1bWVudCIsImFjY291bnQiOiIxazU2YzJjNHhsdGk2X2FjYyIsImdyb3VwIjoid2FkX2NyZXdtYXRlIiwiY29ubmVjdG9yIjoicG9saXNoaW5nX2hlYWQiLCJtZXRob2QiOiJQYmZDYXJkIiwiYmFja3N0YWdlUmVkaXJlY3Rpb24iOmZhbHNlLCJiYWNrc3RhZ2VWaXJ0dWFsTWVtYmVyc2hpcCI6ZmFsc2UsImlkZW50aXR5IjpmYWxzZX0sImlhdCI6MTY0Njc4MjAxOX0.GwupIZSXzUfxSqUuXqRF1RDeRp9dbqRo--HO3GgVKZM

Congratulations, you now can onboard data through the Connect Card! Send it to your customers via email, Slack, or any other communication medium you prefer.

When the customer clicks the link, they will be presented with a Fivetran Connect Card, which will enable them to authenticate the connector, entirely on their own, without sharing credentials with you.

For more information check out the GitHub repository that has sample code we just walked through that you can simply copy, paste and edit for your unique use case.

Learn more about why Powered by Fivetran is the easiest way for your customers to connect data to your app.