How to connect your AI to data

When we wrote about FivetranChat in late 2024, the hard part of building a useful AI tool was plumbing: embedding pipelines, vector search, middleware, and prompt engineering. Two years on, the picture is much different.

The key bottleneck to practical, commercial AI is no longer, and perhaps never was, the power of foundation models. Models are increasingly commoditized, and any off-the-shelf AI can reason through fairly sophisticated questions. However, that doesn't mean AI understands your business. The context a model can access separates a useful answer from a confident hallucination.

Connecting AI to your data therefore requires context engineering — the practice of designing the information and instructions an AI accesses when it generates an answer. AI applications require context infrastructure that constitutes a single, governed source of truth.

With a context layer that labels and explains every data asset, an AI "knows" what it needs to contribute to an organization. More importantly, it brings determinism, provenance, and repeatability to AI output and behavior, while also limiting token usage and computation time.

Practical starter uses of AI include interactive knowledge bases, conversational analytics, and coding copilots that understand your codebase.

[CTA_MODULE]

What is a context layer?

Context is everything needed to interpret data correctly: semantics, business rules, relationships between entities, lineage, and the assorted metadata that explains how a business actually runs.

The capabilities of AI agents and databases fit together hand in glove. AI provides the following capabilities:

• Interpreting questions in everyday language
• Planning multi-step investigations
• Using tools to query data and find documents
• Synthesizing information from across sources
• Explaining findings in plain language

Meanwhile, databases:

• Store and retrieve persistent facts
• Explicitly map out relationships between records
• Join and aggregate large volumes of data
• Run precise, repeatable calculation queries
• Control access through permissions

However, a column labeled "ARR" means little on its own. What makes such a column useful are details such as:

• Where the column comes from
• Which preceding models and operations produce it
• Which accounts it excludes
• Disambiguation from similar metrics

This context lets AI answer practical business questions, such as "What was ARR last quarter, and why did it dip?" 

Context engineering means treating context, including business meaning, as a governed, continuously maintained data product that lives inside the data warehouse itself. Every agent reads the same definitions under the same permissions, instead of inventing its own.

A context layer is where that data product lives. Layered on top of raw data, it becomes the company's digital brain: a shared source of knowledge that people and agents can consult, add to, and systematically use to enhance every analytical and operational use of data.

How to do context engineering

Context engineering can be distilled into four distinct steps:

1. Move
2. Model
3. Describe
4. Serve

1. Move

You need to extract data from the full range of sources — calls, tickets, documents, and tables — and keep it fresh through incremental updates. You will then need to load the data into a single data warehouse or data lake in open formats. We recommend moving data with Fivetran automated connectors. 

This ensures that your AI has timely access to every source in one place, and that there is one version of each fact.

2. Model

Data modeling consists of cleaning, joining, and chunking data, extracting facts, and testing the result. The key is to transform data into models that are trusted, analytics-ready, and can be synthesized into intelligible, actionable insights. We recommend using dbt models to transform data.

3. Define

You will need to systematically map out the meaning, relations, key metrics, rules, etc. represented by your data (as stored in systems of record), knowledge base (as stored in documents), and analytics metadata (such as dbt models and BI semantic layers).

This is the heart of your context layer, and it will read from a data warehouse or data lake. 

4. Connect

Finally, you will need to publish the map you created to a schema and make it available through an MCP.  

We recommend rendering the resulting metadata into plain SQL tables under the open-source Agents Schema standard: one storage location for both structured data and written knowledge. Because it's stored as tables, your context stays under the same governance as the rest of your data. 

Then, using whatever foundation model you prefer, you reach that context in natural language through Agent Context MCP, which exposes a search tool for each connected source. Your front end can be as simple as the desktop app for your LLM of choice with the right extension or skill, plus any additional scaffolding you need.

What can you do with context?

At a minimum, building your company’s digital brain accelerates all the creative, intellectual, and administrative work your organization performs. The immediate, practical wins are anything that surfaces information and needs only read-only access:

• Conversational analytics: Get answers about your business in natural language from your data and documents
• Generative analytics: Generate analyses, visuals, and insights from structured and unstructured data
• AI-assisted data engineering: Build and maintain data pipelines, models, and code with AI that knows your business context

Enterprise AI is still in its infancy. Most organizations sit somewhere near the beginning of the following progression: 

1. AI that makes suggestions
2. Human-in-the-loop workflows
3. Agentic behavior with strict but automated guardrails (i.e., those used for CI/CD)
4. Full autonomy

Agents that act with greater and greater latitude, rather than just answer, will need additional MCPs, APIs, and harnesses. But every stage depends on the same thing — an AI that actually knows your business.

How far we’ve come since the original FivetranChat

In our original post, we described a bespoke retrieval-augmented generation (RAG) architecture for an automated help desk and laid out how you could build one yourself. It was straightforward, but still required considerable engineering know-how: building custom connectors for unstructured data, denormalizing text-rich tables, turning text into embeddings, and spinning up a Slack app to tie it all together. It could retrieve information stored in documents, such as how a table re-sync works, but had no data warehouse connection and couldn’t answer how many customers saw a table re-sync fail last week.

Since then, we have expanded FivetranChat into a broader Fivetran AI platform, adding a robust context layer and conversational analytics capabilities. When someone asks a question, here's what happens under the hood:

1. The foundation model receives the question along with the custom instructions that accompany a skill. Those instructions tell it to break the question into smaller searches, use the relevant tools, and never answer from memory (i.e., its training data) alone.
2. The model calls the context connector on the Agent Context MCP server.
3. Through the MCP server, it examines the context layer — docs, systems of record, and analytics metadata — and uses what it finds to query the data warehouse.
4. It synthesizes the results into an answer, complete with citations.

The instruction against answering from memory matters more than it looks. Foundation models know a great deal about Fivetran from the public internet, much of it outdated. Forcing a lookup keeps answers grounded, and citations let the user check the work.

Step 3 is where the context layer earns its keep. Without it, an agent pointed at a warehouse has to rediscover your schema on every question, burning tokens and guessing at what "ARR" means. With it, the agent starts from definitions your organization already agrees on, and when answering complex questions, reasons from domain knowledge to provide the rationale or stories behind the number. 

After deploying conversational analytics, FivetranChat usage skyrocketed nearly overnight, from about 10% of the company’s headcount in May 2026 to 52% within a month, and climbing. In the same time period, the volume of FivetranChat user messages grew by 6x.

AI usage has spread across every department, where it has been a particular boon to customer support and customer solutions teams, where more than 80% of team members use the tool. SQL queries made by AI now account for almost half of all token usage, far outweighing the original knowledge base use case.

Build the context once

Reusable context keeps AI products straightforward, practical, and reliable. Build your context once, govern it in your warehouse alongside your data, and point every model and agent you adopt at it. When the next project or foundation model arrives, your context comes with you.

[CTA_MODULE]