How to Sync Data from Non-Native Sourceslink
Use Caselink
You want to sync data from a source that Fivetran does not have a connector for, typically a third-party application or vendor. You checked Fivetran’s connector directory and do not see your source listed as a current connector option or as Coming Soon.
TIP: File a "New Connector" Feature Request for your source or see if a feature request already exists.
Fivetran Recommendationlink
Fivetran supports four main workarounds for connecting to non-native sources:
Do regular file exports from your source to cloud storage. Find a list of supported cloud-based storage services in our Files documentation.
If your source application is built on a Fivetran-supported database that you can access, connect to that underlying database. Find a list of supported databases in our Databases documentation.
Use custom cloud functions to pull data from the source API. Find a list of supported cloud functions in our Functions documentation.
Configure webhooks to push data. Learn more in our webhooks documentation.
Search online to see which options work for your source. Some common web searches include the following:
- “[source name] + data export”
- “[source name] + API”
- “[source name] + webhooks”
If you’re not sure which of these options are available for your source, we recommend contacting your source vendor directly, sharing the options above, and asking them which one(s) they support.
Considerationslink
This section describes the pros and cons of each workaround. Depending on your source application, one workaround might be more appropriate than another.
File Exportslink
File exports can generate full file refreshes or delta files for each new file created and delivered to Fivetran. You must pay careful attention to avoid unnecessarily high MAR when setting up file connectors in Fivetran.
Databaselink
Connecting to an underlying database has some added benefits because not only can you securely connect to the database, but you can also track the changes that occur in the source.
Custom Cloud Functionslink
If you want custom cloud functions but don't have technical development resources, reach out to your Fivetran team to discuss the option of working with a consulting partner.
Webhookslink
If an API that we support uses a webhook, we recommend using the API to connect because webhooks circumvents the transformation step of the ETL process. We have not optimized our webhooks connector for a specific source because we built it to be flexible to connect with as many APIs as possible.
We only store the events that we receive after you configure the webhook and cannot fetch historical event data from the web app.
If we don't have a connector for your source but that source has a webhooks API, you can use the source's webhooks to send event data to the Fivetran URL endpoint and we will sync the data to your destination.