AI can build your data connector. Here’s what it can’t do.

Agentic coding tools have changed the story for productivity. Tasks that used to take days now take minutes, and engineers have moved from writing code by hand to spending more time architecting, designing, and scaling. This is a huge win, and moves the entire discipline of software engineering up a level of abstraction — but does it tell the whole story?

Suppose you need to move data from an API, database, or file server into your data lakehouse. The source isn't in any vendor’s connector catalog, but you know that AI can generate code fast. The obvious move is to point Claude Code or Cursor at the docs and have it start writing code.

But the connector you get won’t be production-ready.

[CTA_MODULE]

What building a connector actually involves and the limits of pure DIY

Getting data from an API into a cloud destination reliably means more than extracting and loading data, and includes:

• Authentication and token refresh
• Handling pagination and rate limits consistently
• Managing incremental syncs so you're not pulling everything on every run
• Detecting and adapting to schema changes at the source
• Retry logic for transient failures
• Orchestration and scheduling
• Alerting when something breaks
• Testing the connector is moving and updating the data correctly

A data engineer with an AI coding agent can get the basics written in an afternoon. Tools like dlt make it even faster to scaffold a full pipeline, and developers genuinely love this workflow: it's fast, hands-on, and the first sync feels like a win. The problem is that you own everything that comes after it. Managing the infrastructure it runs on, schema drift, validation, API version changes, retry logic, authentication failures, and incremental sync state: all of it lands on your backlog. Even with the help of AI, your engineers must dedicate time to testing and maintenance. 

Open-source tools still require significant engineering effort

Platforms like Airbyte and Meltano add structure to the pipeline and come with growing connector catalogs, sometimes offering AI-assisted connector generation. 

However, self-hosting still means you own the infrastructure: provisioning, scaling, monitoring, and incident response are yours. AI generates the connector code faster than an unaided human engineer, but it doesn't change who gets paged when the pipeline fails. 

Connector quality across community-built catalogs is also uneven; many connectors are maintained by contributors rather than the vendor, so reliability and incremental sync support vary by source. When infrastructure and engineering time are included, the total cost of ownership lands closer to a solution like Fivetran than the open-source label suggests.

What Fivetran adds

Fivetran has 750+ native connectors, covering most of your likely data sources. However, there will always be the odd custom source outside of the catalog, tempting you to resort to DIY. 

When you do hit this scenario, Fivetran offers options depending on your needs, timelines, and the level of control you want over the logic. The custom connector options listed below all run on Fivetran's managed infrastructure. The solved problems of orchestration, scheduling, data writing, and retries are Fivetran's responsibility, regardless of which you choose. And Fivetran uses AI tools internally to speed up both native connector development and custom connector development for you.

I have simple needs but no bandwidth to code, with or without AI

Your first port of call should be our AI Connector Agent (currently in beta); this generates an ingestion-ready connector in minutes from your source's API documentation and is the place to start if your source provides a REST API for the data you need:

• Provide the API docs URL (HTML, PDF or Swagger) then select which endpoints to sync, approve the schema, and the data starts moving
• No code, deployment, or infrastructure
• Supports REST APIs with JSON responses and standard auth: API key, HTTP Basic, Bearer tokens, and OAuth 2.0
• If something needs updating, simply raise a support case or create a new connector

This choice is ideal if your source has a well-documented REST API, you want to sync data immediately, and you don’t want to write or own connector code. Consider this option first if you need a custom connector. 

If the AI Connector Agent doesn’t initially produce the results you want then reach out via our support team to have our engineering team review and address any issues. Here we work with your team to iterate on your previously generated connector to get it working as you intend:

• You provide context on the use case and data source; Fivetran leverages the AI generated connector to accelerate development of a native Fivetran connector
• Fivetran does initial testing of the connector
• The connector goes through a private preview phase with you for testing and feedback before becoming available to all Fivetran customers
• 5 to 6 weeks to review the request, then, if accepted, the typical development time is 6 to 8 weeks
• Requires a well-documented API; sources with custom data elements specific to your implementation aren't supported

This is the best choice if you need a connector, prefer not to write code, and are willing to spend time working with Fivetran to get it right.

I have complex needs that require coding, but AI can help me!

Connector SDK is our Python package that allows an AI-assisted engineer or coding agent to write a Fivetran connector. You own the connector logic and the maintenance, while Fivetran owns all the infrastructure underneath it to execute. This is also what you’ll need to use if your source isn’t REST based as the options only support REST. 

• You and your coding agents write the connector in Python
• Supports practically anything Python does - APIs, REST, XML, files, databases, and more.
• Fivetran provides skills and plugins for AI coding tools such as Claude Code, Codex, Gemini, GitHub Copilot, and others
• Fivetran also has a repository of 100+ Community Connectors for Connector SDK, so a foundation for your source may already be there
• What used to take weeks now typically takes hours to days with AI assistance
• You maintain the connector for changes needed to your custom logic, or sign up for Connector SDK Long-Term Maintenance
• Fivetran offers help for current customers via the “Save Time Now” program. This is time-limited free assistance to get started with Connector SDK. We can work together over screen-share sessions or emails.

The following table summarizes the characteristics of each choice:

What’s next?

AI has changed the paradigm around the required technical skills and the speed of development for ingesting data from custom sources, but it hasn’t solved the last-mile problems related to infrastructure and maintenance. We know that AI can speed up the development of connectors as we use it internally both when creating our native connectors and when working with customers. We also know that AI doesn’t write production-grade connectors on its own, which is why we have built a platform with extensive scaffolding and infrastructure around it to ensure its success.

Between AI Connector Agent, support team assistance, and Connector SDK, Fivetran offers the versatility and customizability to handle edge and corner cases while guaranteeing reliability and time-to-value by leveraging our infrastructure.

If your source has a well-documented REST API, then try the AI Connector Agent first and get results in minutes. If the AI Connector Agent doesn’t work for your REST API, then reach out via Fivetran Support. For non-REST sources, complex replication, and filtering needs of all kinds, get started with the Connector SDK.

[CTA_MODULE]