Why NLWeb Isn't Enough — And What We Actually Need for the Agentic Internet

December 19, 2024

Everyone's excited about NLWeb — the open protocol from Microsoft that promises to make websites discoverable and usable by AI agents. It's clean, elegant, and developer-friendly. It works by letting websites expose a .well-known/agent.json file and a natural language /ask endpoint, so agents can query them and receive structured responses. Think of it as SEO for agents.

But here's the uncomfortable truth: NLWeb won't work for the most important use cases. And unless we rethink the deeper layers of agent interaction, we'll repeat the same mistakes of the early web — building tools that feel good on paper but collapse at scale.

The Real Problem: Capabilities, Not Content

NLWeb is great for publishing public, semi-static information — job listings, events, product catalogs, documentation. But as soon as you enter the realm of real transactional workflows — like booking a hotel, managing a bank account, filing an insurance claim — NLWeb falls apart.

Why? Because no serious company will expose those workflows over an open query endpoint. Imagine Airbnb opening its search, booking, and payment flows through a /ask API exposed to every agent. You'd see:

•Competitive wrappers extracting value without attribution
•Loss of control over UX and onboarding
•Potential API abuse and brand erosion

In practice, the workflows that matter most require trust, identity, and boundaries. These aren't just content queries. They're secure, multi-step processes — and that's not what NLWeb was designed for.

The Right Model: Agents Talking to Agents

Instead of exposing raw endpoints or unguarded data, what companies actually need is a way to expose capability-bound agents. Not "here's our backend," but:

"Here's our official booking agent. Talk to it. It knows how to search, price, and book — under our rules, with our data, behind our auth."

In this model:

•My agent (on my phone or browser) sends a request to Airbnb's agent.
•Airbnb's agent executes the action internally.
•I never touch their APIs directly.
•They never expose anything they don't want to.

This flips the model from agent-to-website to agent-to-agent — and it's where protocols like MCP and A2A (in theory) start to make more sense. But even those aren't ready yet. They lack registries, permission systems, economic flows, and hosting standards.

What We Actually Need

We don't just need agents. We need the infrastructure to support them:

✅A way for companies to build and deploy controlled agent endpoints (with prompts, memory, tools, and vector-backed context)

✅A discovery mechanism — whether centralized, federated, or decentralized

✅A permission layer: What's public? What requires auth? What can be charged for?

✅A payment and metering protocol — so actions can have value

✅A standard UI layer for agents (for humans and agents to co-interpret)

Right now, none of this exists. NLWeb solves the read-only, public side of the agentic web. But the future is not read-only. It's transactional, interactive, and tightly scoped.

The Next Layer

What I'm proposing is a shift in thinking:

✗Not websites becoming agent-readable

✓But companies becoming agent-expressive

Every business will expose a set of agents — just like they once built websites, then APIs, then apps. The protocol we need will let agents be safely hosted, discovered, and interacted with — while keeping control in the hands of the business.

NLWeb might be part of the answer. But if we stop there, we'll never get the agentic internet we actually need.