BI Tools Are Holding Your AI Strategy Back

I have been talking with a lot of companies about becoming AI-first.

The conversation usually goes something like this:

We want to use AI across the data team. We just do not want to change any of our tools.

I understand why. Companies have spent years implementing Power BI, Looker, ThoughtSpot, dbt, a data catalog, a task manager, and everything around them. People know how the tools work. The contracts are signed. The dashboards already exist. Nobody wants to volunteer for another migration.

So the natural plan is to wait for every existing vendor to add AI.

Power BI gets Copilot. Looker gets Gemini. ThoughtSpot gets Spotter. Dashboard creation gets faster. A business user can ask a question in plain English. An analyst can generate a chart without clicking through every menu.

All of that is useful.

It is also probably not what it means to become AI-first.

We have seen this movie before

The evolution of AI in coding is a useful analogy.

The first big improvement was autocomplete. You started typing a line of code, and the model guessed the rest. Then it could generate a function. Then it could explain an error or suggest a test.

This made the existing workflow faster. The developer still decided what file to open, carried the context, made the edits, ran the tests, read the failure, and figured out what to do next.

Autocomplete was valuable. But it did not change the unit of work.

Coding started to feel fundamentally different when the unit of work became the task.

Now I can open a terminal and say: investigate this bug, understand the codebase, make the change, run the tests, inspect what failed, fix it, and bring me back something ready to review.

The agent is not predicting the next line. It is operating a loop:

understand → investigate → build → test → inspect → revise → validate
     ↑                                                     ↓
     └──────────────────── keep going ─────────────────────┘

I still provide judgment. I still review the result. But I am no longer responsible for manually moving every intermediate step forward.

OpenAI describes this shift as changing the unit of knowledge work from a short interaction to a delegated, long-horizon task. Its own employees increasingly use Codex for work that can run for minutes or hours, across engineering and nontechnical teams alike (OpenAI).

That is the change data teams should be looking for.

Most AI in BI is still autocomplete

The current generation of BI products is becoming much easier to use.

Microsoft says Copilot in Power BI can answer questions about a report, summarize pages, write DAX, and generate a report page from a prompt. Looker's Conversational Analytics lets users query Explores and dashboards in natural language, with authored instructions and LookML as grounding. ThoughtSpot says Spotter can break questions into steps, test assumptions, rerun analysis, and trigger actions in other systems.

These are meaningful improvements. They are not all equivalent, and it would be lazy to pretend that the products are standing still.

But here is the test I care about:

What happens when the work crosses the boundary of the BI tool?

Suppose an executive asks why revenue is wrong on a dashboard.

The answer may be a hidden filter. It may be a broken definition. It may be a many-to-many join in a transformation. It may be a source that stopped updating. It may be an API whose schema changed. It may be a known tracking problem that someone discussed in a meeting and already scheduled for repair.

If the AI can only generate a better chart, query a semantic model, or summarize what is already on the screen, the human still owns the real work.

The human has to move from the dashboard to the metric definition, from the metric to the model, from the model to the raw data, from the raw data to the ingestion job, and from the ingestion job to the task and the conversation that explain what changed.

The dashboard step may now take five minutes instead of an hour. The investigation may still take three days.

That is autocomplete for data work.

The dashboard is not the outcome

This is the part I think many AI strategies get wrong.

They start with the existing tool and ask, “How can AI make this tool faster?”

That question almost guarantees a local improvement. The dashboard tool makes dashboards faster. The transformation tool makes SQL faster. The catalog makes definitions easier to find. The task manager summarizes the tickets.

You end up with smarter tools and the same fragmented system.

I wrote about this as the difference between local AI and global AI. Local AI makes each application smarter. Global AI is designed around the entire outcome.

For a data team, the outcome is rarely “produce a dashboard.”

The outcome is something like:

• ✓Decide what the business needs to understand.
• ✓Turn that question into a clear piece of work.
• ✓Define the business concepts and metrics.
• ✓Model the underlying data.
• ✓Build the analysis or application.
• ✓Test the logic and the data.
• ✓Deploy it.
• ✓Inspect what a user actually sees.
• ✓Validate the numbers against the source.
• ✓Diagnose anything that looks wrong.
• ✓Record what changed and what still needs work.
• ✓Keep going until the result is trustworthy.

That flow moves forward and backward. Testing changes the model. A strange chart sends you back to the definition. A fresh source reveals that an old assumption no longer holds. A user's question becomes a new task, which becomes a transformation, which changes the dashboard, which needs another round of validation.

AI-native data software must be able to operate that loop.

It cannot stop at the edge of the canvas.

A real data agent needs the entire chain

This does not mean dumping the warehouse, every Slack message, and the entire company wiki into one enormous prompt.

It means giving the agent a coherent map and the ability to retrieve and act on the right context as the work unfolds.

To debug one dashboard number, the agent may need:

the user's request and current task
                ↓
dashboard state, filters, and visual output
                ↓
metric definitions and business rules
                ↓
models, transformations, tests, and lineage
                ↓
job history, freshness, and source behavior
                ↓
raw data, incidents, decisions, and prior work

Then it needs to move back up the chain: fix the issue, rerun the work, deploy the result, inspect the dashboard, and verify that the number is now correct.

I explored the debugging side of this in The Data Stack AI Problem. The key point is that the truth does not live in the dashboard. It lives in the connections among the dashboard, definitions, code, source data, system state, and organizational knowledge.

An AI that can see only one piece can help with that piece. An AI that can traverse the system can own the outcome.

The leading AI companies are building loops, not shortcuts

Look at how the companies building frontier AI use AI themselves.

OpenAI says researchers use GPT-5.6 across a development loop that includes diagnosing failures, optimizing training systems, running experiments, and interpreting results—not merely writing the next line of research code (OpenAI).

It also uses AI to examine the behavior of coding agents in real internal deployments. The monitoring system reviews conversations, tool calls, and outputs, surfaces problematic behavior for human review, and feeds what the team learns back into mitigations and stronger safeguards. That is production behavior becoming input to the next improvement loop—not a team waiting for someone to notice a bad dashboard (OpenAI).

Its GPT-Red system makes the pattern even clearer. The model attempts an attack, observes how another model responds, adjusts, and tries again. OpenAI then uses the failures it finds to make GPT-5.6 more robust. The valuable part is not one clever response. It is the closed loop from finding a weakness to improving the production system (OpenAI).

OpenAI's internal data agent works the same way. It goes from a question to data discovery, queries, analysis, and published outputs. If an intermediate result looks wrong, it investigates, changes its approach, and retries. The system carries the context forward instead of making the user restart the analysis at every step (OpenAI).

Anthropic keeps arriving at the same architecture from different directions. Its guide to building effective agents emphasizes that coding agents are powerful because tests create feedback the agent can use to iterate. Its work on long-running agent harnesses found that agents need persistent state, incremental progress, and explicit end-to-end testing—not a giant prompt and a one-shot answer. In a more extreme experiment, Anthropic put Claude instances in a continuing loop and used failing tests to guide them toward a compiler that could build the Linux kernel (Anthropic).

The lesson is remarkably consistent:

The value of AI appears when the system can observe the result of its work and decide what to do next.

Data should be no different.

Why we built Polyform in one place

When I started building Polyform, I seriously considered assembling it from the same collection of tools everyone else uses.

There were good products for ingestion, modeling, metrics, BI, task management, and orchestration. Building each part ourselves looked unnecessarily ambitious.

The reason we ultimately built the system end to end was context.

I wanted the definition to remain connected to the model. I wanted the model connected to the source and its processing state. I wanted the dashboard connected to the exact metric and transformations behind it. I wanted the work itself—what someone requested, what changed, what failed, and what still needed review—to live in the same operating environment.

Most importantly, I wanted the AI to be able to move through all of it.

As I described in My Journey to an AI Agent That Actually Works, we eventually made the whole platform legible to an execution agent. It can investigate, write code, operate the product, test its work, inspect the result, and continue until it has something useful to bring back.

The goal was never to win a feature checklist against every specialized tool.

The goal was to create the loop.

Do not confuse preserving your stack with becoming AI-first

You can absolutely make Power BI, Looker, or ThoughtSpot part of an AI-native system.

But adding their AI features does not make the surrounding system AI-native by default.

Ask a harder question:

Can the AI take a data problem from request to trusted result—including the definitions, models, tasks, tests, deployment, inspection, and revision—or does it hand the work back to a person when it reaches the edge of the dashboard?

If it hands the work back, you have better autocomplete.

That may save time. It may be worth buying. It may make many people more productive.

But it is not the transformation we saw in coding, and it is not the transformation data teams should settle for.

The future of data is not a faster way to build the same dashboard.

It is a system that understands why the dashboard needs to exist, builds everything required to make it trustworthy, sees when the result is wrong, and keeps working until it is right.