Control plane vs. data plane: How they work in a hybrid data architecture

Data integration platforms treat pipeline management and data movement as two separate jobs. The control plane covers the management side, and the data plane handles the movement of records.

The location of each plane matters most in a hybrid architecture. Data engineering teams often want a vendor to handle pipeline setup and monitoring for reliability and reduced operational overhead. At the same time, security policies require regulated data to stay inside the company’s cloud account or on-premises network. A hybrid setup meets both needs with a vendor-run control plane and a data plane inside the company network.

This article provides a comparison of the control plane vs. data plane in a hybrid setup and explains how Fivetran’s Hybrid Deployment keeps pipeline processing in your environment.

What is the control plane vs. data plane?

The control plane is the part that handles configuration in a distributed system and decides what work happens and when. The data plane is the part that performs the jobs the control plane schedules, such as pulling records from a source system and writing them to a destination.

In a pipeline platform, creating a connection or changing its sync frequency is a control plane action, while extracting rows from PostgreSQL and loading them into a warehouse counts as a data plane action.

Both terms come from networking, where routers use the control plane to decide the best path for traffic and the data plane to send packets along that path.

Many modern distributed systems follow the same pattern but run the two planes on separate machines. For instance, in software-defined networking, a centralized controller makes routing decisions, while switches handle forwarding. In Kubernetes, the control plane assigns containers to servers, and the worker nodes run them.

Here’s a side-by-side comparison of the data plane vs. control plane:

How do the control plane and data plane work in a hybrid data architecture?

In a hybrid data architecture, the control plane runs as a managed cloud service, and the data plane runs inside an environment your organization controls, such as a VPC or an on-premises data center.

This setup lets data engineers manage every pipeline from one centralized interface, while extraction and loading happen close to the sources and destinations. Data architects choose a hybrid setup when security rules prevent regulated sources from sending raw records to third-party servers. On-premises SAP systems and databases with customer personal information are common examples where hybrid cloud data integration makes sense.

Here’s how it works.

The control plane manages the system

The control plane keeps a record of how each pipeline is supposed to run. It stores connection settings and sync schedules, decides when each job starts, enforces user roles and access policies, and collects the health signals behind alerts and dashboards.

For example, if your analytics engineer changes a Salesforce connector to sync every hour instead of every eight hours, the control plane saves that update and starts scheduling hourly runs. The engineer doesn’t have to log in to the server that runs the data plane. Since the vendor runs and patches a managed control plane, your platform team doesn’t need to maintain any scheduling or monitoring servers.

The data plane handles the data work

The data plane moves the actual records during each sync. It connects to the source database or API, pulls new and changed records, stages them in temporary storage, and writes them to the warehouse or data lake.

In a hybrid deployment, all of this work — the source connection, record extraction, temporary staging, and destination writes — happens inside your network boundary. Raw records stay in the regions and accounts your compliance team has approved. A data plane running inside your network matters most when your team needs to replicate sensitive databases that hold patient records or payment card data.

The two planes still communicate

The two planes still talk to each other, but the communication is limited to operational details. Configuration and run schedules flow from the control plane, and the data plane reports back with run status, row counts, sync metrics, and logs.

The data plane keeps customer records inside the company network and never sends them to the control plane. Pipeline health dashboards rely solely on run status and sync metrics, with no access to the underlying records.

Why the control plane vs. data plane separation matters

The separation helps data engineering teams in several ways, especially during audits and outages:

• Data residency: Raw records stay in the cloud region or data center where the data plane runs, which simplifies audits under rules like the General Data Protection Regulation.
• Central management: Data engineers can configure and monitor on-premises and cloud pipelines from one interface, with a single scheduler and alerting setup covering every environment.
• Network boundaries: A data plane that runs behind your firewall and opens outbound connections keeps source systems unreachable from the public internet.
• Resilience: A failure in one data plane stops only the pipelines it runs, so syncs in other data centers and cloud accounts can continue.

The data residency and network protection benefits hold up only when the hybrid pipeline tool runs the data plane inside your network and the vendor’s control plane receives nothing beyond metadata and logs.

How Fivetran uses control and data planes in Hybrid Deployment

Fivetran uses the control plane and data plane model in Hybrid Deployment, where a fully managed cloud service runs the control plane and a Hybrid Deployment Agent runs the data plane inside your environment.

A sync moves through Hybrid Deployment in four steps:

1. Configure the connection. Data engineers create a connection in the Fivetran dashboard or API and assign it to a registered Hybrid Deployment Agent.
2. Receive run instructions. The agent, deployed on Kubernetes clusters or a Linux host with Docker or Podman, checks in with Fivetran to get sync schedules and job details.
3. Process data locally. The agent runs each sync inside your network, extracting from the source and loading into the destination, with temporary files held in storage buckets your team owns.
4. Report back for monitoring. The agent returns sync metrics and logs to Fivetran, allowing your team to track every pipeline’s status in the dashboard.

With the Hybrid Deployment Agent running syncs inside your network, enterprise data teams get several advantages:

• Local handling of sensitive data: Regulated sources, such as SAP ERP on HANA and Epic Clarity, sync without any records passing through Fivetran’s infrastructure.
• One dashboard for all pipelines: Data engineers manage cloud-hosted and locally hosted pipelines side by side, using the same API and Terraform workflows for both.
• Outbound-only connections: The agent initiates every mutual Transport Layer Security connection to Fivetran from inside the private network, keeping inbound firewall ports closed.

With Fivetran, data engineers can keep regulated data inside your network while managing every pipeline from one place.

Start a free trial to see how Hybrid Deployment works with your sources.

FAQ

What is a control plane?

A control plane is the management side of a distributed system. It exposes the administrative APIs that create, read, update, and delete resources. Control planes handle requests, such as pausing a data pipeline or changing its sync schedule, before the data plane moves any records from source to destination.

Why do organizations separate the control plane and data plane?

Separation allows each plane to scale independently, which matters because the data plane often handles far more requests than the control plane does. The split also allows platform engineers to update management features without interrupting the movement of production data.

‍

[CTA_MODULE]

‍