All articles
SEO

The Agentic Internet: How AI Agents Are Changing the Way People Search

Bots now make more web requests than people, and AI agent traffic grew more than 1,700% in a year. Here's what Cloudflare's data says about how search is changing, and what it means for small business websites.

WebTempo Team6 min read
A web page with its business name, price, hours and booking button highlighted, each passed as a tag to an AI agent

Your website has a second audience, and it's growing fast.

In May 2026, automated traffic passed human traffic on Cloudflare's network for the first time. According to Cloudflare's 2026 Founders' Letter, the company had expected that to happen in the second half of 2027. It arrived more than a year early, pushed along by AI crawlers and, increasingly, AI agents: software that browses the web on someone's behalf to compare prices, read menus, check availability, and get things done.

Cloudflare sits in front of more than 20% of the web, so its numbers are one of the clearest views we have of how this shift is playing out. Here's what they show, how it's changing the way people search, and what small businesses can do about it. (For what we heard in person, see our takeaways from Cloudflare Connect 2026.)

What "agentic" means

For most of the web's history, two kinds of visitors mattered: people, and search engine crawlers that read your pages so they could send people back to you.

AI has added two more:

  • AI crawlers read pages to train AI models or to answer questions in tools like ChatGPT, Claude, Perplexity, and Google's AI Overviews.
  • AI agents act for a person in real time. Ask an assistant to find a dentist who takes your insurance and is open Saturday, and it may read a dozen websites, compare them, and hand back one answer. It might even book the appointment.

That second group is the one growing fastest. Cloudflare reports that daily requests from AI agents on its network grew by more than 1,700% over the past year.

The numbers behind the shift

A few figures from Cloudflare's 2025 and 2026 reporting:

  • More than half of web traffic is no longer human. Cloudflare says 2026 is the first year that's been true. Its network has almost doubled since the end of 2024, from an average of 63 million requests per second to about 115 million, with peaks above 150 million, and the growth of agents and AI crawlers is why bots overtook people more than a year ahead of Cloudflare's forecast.
  • AI crawlers take far more than they give back. Cloudflare measures how many pages each AI company reads for every visitor it sends to a website. In July 2025, Anthropic read about 38,000 pages per referral, OpenAI about 1,100, and Perplexity about 195. Google, by comparison, read about 5 pages for every visitor it sent.
  • More crawling is for training, not for sending visitors. By the crawlers' own stated purpose, AI training rose from 22% of crawler requests in spring 2025 to 52% by June 2026.
  • Agents acting for people are taking off. Crawling triggered by a person's request, such as an assistant fetching a page to answer a question, grew more than 15 times over during 2025.
  • People are visiting less. In some of the most heavily crawled industries (retail, software, IT services, and financial services), human traffic fell by as much as 40% in less than a year.

How this is changing search

For 25 years, the deal was simple: let search engines read your site, and they'll send you visitors. Cloudflare argues that bargain is breaking down. Answer engines now summarize what they read, so the reading costs you bandwidth without bringing people back.

You can see it in search behavior too. When Pew Research Center tracked real people's Google searches in March 2025, users clicked a regular result in 8% of visits when an AI summary appeared, versus 15% when it didn't. Only 1% clicked a link inside the summary itself, and people ended their browsing session on 26% of pages with an AI summary, compared with 16% without one. Cloudflare saw the same pattern from the website side: Google referrals to news sites fell 15% between January and April 2025, as AI Overviews expanded.

Search is shifting from ranking to recommending. Instead of scanning ten blue links, more people ask a question and get one answer, often assembled by an agent that read dozens of sites to produce it. Cloudflare's founders put it this way: an agent might read 1,000 menus to recommend one restaurant.

Why this matters for small businesses

This is where it gets personal for local businesses. In Cloudflare's 2026 Founders' Letter, Matthew Prince and Michelle Zatlyn point out that small businesses win customers with emotion or convenience. Think of the person behind the counter who remembers your name, or the shop on your way home.

Agents don't weigh those things. They tend to favor whoever has the most information online, which is usually whoever has been around the longest. Left alone, that makes it harder for newer and smaller businesses to get recommended.

The flip side is an opportunity. An agent can only recommend what it can read and understand. A small business with a fast, clear, well-organized website can be easier for an agent to recommend than a bigger competitor whose key details are buried in PDFs, images, or pages bots can't reach.

What to do about it

You don't need to rebuild your marketing around AI. Most of what helps agents also helps people:

  1. Make sure AI tools can actually read your site. Overly strict bot protection or a misconfigured robots.txt can lock out the very assistants your customers use. Fewer than 1% of sites on Cloudflare block search crawlers, but 17% block AI training crawlers, and it's easy to block more than you meant to. Since July 2026, Cloudflare has let every customer, including those on the free plan, treat search, agents, and training separately, so you can keep showing up in answers without handing your content over for training.
  2. Put your key facts in plain text. Hours, prices, services, service area, menus, and policies should be real text on your pages, not locked inside images or downloadable PDFs.
  3. Answer the questions people actually ask. "Do you take walk-ins?" "How much does a website redesign cost?" "Do you serve St. Paul?" Clear answers on your site give an agent something specific to repeat.
  4. Keep your details consistent everywhere. Your website, Google Business Profile, and directory listings should match. Agents cross-check sources, and conflicting hours or addresses make you look less reliable.
  5. Keep your site fast and well built. Agents read thousands of pages. Clean, quick pages with good structure are easier to read and quote correctly.
  6. Watch where your traffic comes from. Check your analytics for visits from ChatGPT, Perplexity, and other AI tools. They're small for most sites today, but they're the trend to watch.

See how your site holds up

We built our free website audit with this shift in mind. In about a minute, it checks your speed on phones and computers, your search basics, and whether AI tools like ChatGPT, Perplexity, and Claude are allowed to read your site, then tells you in plain English what to fix first.

If you'd rather have someone look it over with you, get in touch. Getting found by people and by the agents working for them is exactly what we help small businesses with.

Sources

Related tags

Ready To Build Something Remarkable?

Great websites don't happen by accident. They're built with intention, tested for speed, and designed to deliver.
Let's build together
Or run a free audit of your current site
© 2026 WebTempo LLC. All rights reserved. | Twin Cities digital design, SEO, & hosting experts
Privacy Policy