The Grounding Layer Every Agentic AI Project Needs (Or Joins the 40% Gartner Says Get Cancelled)

In June 2025, Gartner published a prediction that landed like a brick on every CTO desk in enterprise B2B SaaS: by the end of 2027, more than 40% of agentic AI projects will be cancelled · due to escalating costs, unclear business value, or inadequate risk controls.[1] In our view, much of that risk comes down to grounding: the AI's ability to anchor its responses in verifiable, current, traceable data instead of confabulating answers from training data.

Seven months earlier, in November 2024, Anthropic open-sourced the Model Context Protocol — MCP · an open standard for connecting AI models to external data sources at inference time.[2] By mid-2025, OpenAI, Google, GitHub, and Microsoft had all adopted MCP in their assistant and copilot offerings. The architectural pattern the industry converged on is no longer ambiguous: agentic AI ships with a grounding layer or it gets cancelled.

Gartner predicts more than 40% of agentic AI projects will be cancelled by the end of 2027. The industry's architectural answer is converging on typed knowledge graphs connected through MCP.

This post is about why the convergence is happening, what the Microsoft Research GraphRAG benchmarks actually show about graph-grounded vs document-grounded retrieval, what MCP gets right architecturally, and what every CTO needs to know about the grounding-layer substrate decision over the next 12 months.

Section 1: The grounding gap, by the numbers

The cost of AI going wrong is now documented in primary research. EY's 2025 Responsible AI Pulse survey of 975 C-suite leaders found that 99% of their organizations had suffered financial losses from AI-related risks, with an average loss of about $4.4M among those affected.[5]

Stanford's 2025 AI Index Report tracks the technical side. It finds that, despite real progress, LLMs still produce false information that appears credible, and that the harder factuality benchmarks introduced in 2024 remain difficult for leading models.[4]

And the regulatory backstop is now real. The EU AI Act's Article 50 transparency obligations apply from August 2026, with fines of up to €15M or 3% of worldwide annual turnover for non-compliance.[7] Article 50 doesn't prescribe an architecture, but being able to show where an answer came from is far easier on top of a knowledge graph than on top of a vector store.

Section 2: What MCP gets right architecturally

Anthropic's Model Context Protocol is the architectural move that signaled the industry consensus.[2] It's worth being precise about what MCP is and what it isn't.

What MCP is

• An open standard for connecting AI models to external data sources at inference time.
• A typed protocol: tools expose resources (queryable data), tools (callable functions), and prompts (templated workflows). The model can introspect each.
• Vendor-neutral: Claude, ChatGPT, GitHub Copilot, Microsoft Copilot Studio, and a growing list of internal enterprise LLMs all speak it.

What MCP isn't

• A data source itself. MCP is the wire format; you still need a substrate behind it that actually holds the typed, grounded, traversable data.
• A retrieval algorithm. MCP doesn't solve the multi-hop reasoning problem — it surfaces what the underlying data source returns. If the source is a vector store, you get vector retrieval through MCP. If the source is a knowledge graph, you get graph traversal through MCP.
• A solution to grounding by itself. MCP is necessary infrastructure; the grounding substrate is the decision MCP makes possible · not the decision MCP makes for you.

Section 3: Why the substrate behind MCP needs to be a graph

Microsoft Research's April 2024 GraphRAG paper is the best-known head-to-head comparison on this substrate decision.[3] The paper introduces a clean distinction between two question types:

• Local questions: “What did this specific document say about X?” — the retrieval problem vector RAG was designed for. Both substrates perform well here.
• Global questions: “Across the entire corpus, what are the recurring themes about X? What nodes recur? What relationships hold?” — the multi-hop reasoning enterprise agents actually need.

On global questions, an LLM judge preferred GraphRAG's answers to baseline vector RAG on comprehensiveness in 72-83% of head-to-head comparisons.[3] The reason: vector search returns semantically similar passages but cannot recognize when two passages refer to the same thing, or when one thing is the parent of another. GraphRAG builds an explicit graph of those things and traverses relationships before returning context. Microsoft has since released GraphRAG as open source, aimed at discovery over private, narrative data.[9]

For an agentic AI system, the practical translation is straightforward:

• Vector substrate behind MCP: agent can answer “what did the security policy doc say?”
• Graph substrate behind MCP: agent can answer “which competitors shipped feature X in the last 30 days, and which of our strategic accounts have asked for it?”

Almost every enterprise question that justifies the cost of building an agent is the second kind.

Section 4: The four properties grounding requires

Distilled across the Gartner prediction, the EY loss numbers, the EU AI Act compliance burden, and the Microsoft GraphRAG benchmarks, a grounding layer that actually keeps an enterprise agentic AI project alive needs four properties:

1. Typed nodes, not just text chunks

Every relevant concept · vendor, product, customer, contract, capability, regulation — needs to be a typed node with a stable identifier. Text chunks are not nodes. Vector embeddings of text chunks are not nodes. The substrate has to know that “Salesforce” in one place and “SFDC” in another refer to the same vendor node.[8]

2. Provenance on every claim

Every fact in the graph must carry a source URL, a retrieval timestamp, a confidence band, and a model + version identifier. The agent's response must be able to surface this provenance on demand. Without provenance, nobody can check where an answer came from.

3. Multi-hop traversal as a first-class operation

The agent must be able to follow edges across multiple node types — vendor → product → capability → competitor → release → review → sentiment — without losing precision. Vector similarity at each hop accumulates error. Graph traversal does not.

4. Tenant isolation by construction

For enterprise customers, the customer's private graph slice (their accounts, their contracts, their internal feature taxonomy) must never cross into the public graph or another tenant's slice. Row-level security at the database layer, not application-layer filtering. The cost of getting this wrong on a regulated dataset is unbounded.

Typed nodes. Provenance on every claim. Multi-hop traversal. Tenant isolation by construction. Those are the four properties. The substrate that natively has all four is a knowledge graph.

Section 5: The two architectural paths

For an enterprise CTO making the substrate decision today, the practical fork:

Path A: build the grounding layer internally

Build a Neo4j or RDF-based knowledge graph against your internal nodes (your products, your customers, your contracts) plus a curated set of external nodes (your top 50 competitors, your top 30 categories). Wire it behind an MCP server. Iterate. Public references for what this looks like at scale include LinkedIn's Economic Graph, Google's Knowledge Graph, Microsoft's Project GraphRAG.[9][8]

This path works, but it takes many months of engineering and data acquisition before the first useful production query, plus an ongoing maintenance team.

Path B: subscribe to a productionized market graph + integrate internal nodes

The external side of the graph · every competitor, every category, every signal — is the part that's expensive to maintain and where economies of scale are real. Subscribing to a productionized market graph means you only own the internal side of the schema: your accounts, your products, your contracts. They integrate into the same graph schema with tenant isolation enforced at the database layer.

The agent and the MCP wire format stay the same; what changes is that the expensive half of the graph already exists.

Section 6: The cancellation risk, concretely

In our view, the projects that get cancelled tend to fail in three recognizable ways:

• Hallucination beyond tolerance: the agent confidently fabricates competitor capabilities, customer history, contract terms. Stakeholder trust collapses. Project gets paused, then quietly killed.
• Compliance pushback: legal or compliance teams refuse to approve the agent for customer-facing or regulated use because there's no audit trail. Compliance review stalls. Project gets descoped to internal-only, then defunded.[7]
• Cost-to-value collapse: ungrounded LLM inference is expensive and the answers aren't good enough to justify the spend. The CFO asks for ROI. The team can't produce a defensible number. Project gets cut at the next budget cycle.

Every one of these patterns is structurally a grounding problem. Every one of them is solved by moving the substrate behind MCP from a vector store to a knowledge graph.

Where to go from here

The next year or two is the substrate-decision window for enterprise agentic AI. The companies that move first to a graph grounding layer behind their MCP integration are going to be the ones whose agentic AI projects ship and stick. The companies that stay on vector-RAG substrates because the demo worked are likely to be the projects Gartner counted in the 40%.

The substrate decision is upstream of the agent decision, the workspace decision, the copilot decision. For the next CTO conversation about agentic AI strategy, the question worth asking before any other question: “What's our grounding substrate — and is it a graph?”