AI Agent Traffic Becomes a Cost for Websites
Website operators facing agent scraping must weigh blocking, rate-limiting, or monetizing that traffic against their infrastructure spend.
Updated on Nov. 6, 2026 in Public Companies
Live Poll
Do you think AI agents crawling websites is a fair cost for businesses to bear?
Cloudflare reported record quarterly revenue of $782.5 million, up 39 percent year-on-year, and CEO Matthew Prince warned on the third-quarter earnings call that AI agents are a bane to some customers' businesses. The company's shares rose around 15 percent in after-hours trading as major US indices fell about four percent that Thursday.
Why it matters
Some Cloudflare customers see agentic traffic as imposing costs without benefit, a pressure point for any business whose website and content are scraped by automated tools. The comments come as Cloudflare says roughly five percent of the web has opted into its content protection products since July 1.
Cloudflare reported a record $782.5 million in quarterly revenue, up 39 percent year-on-year, and said approximately five percent of the web has opted into its content protection products since July 1. The company projects bot-to-human traffic could reach a 1000:1 ratio, though the source does not give a measured baseline for that figure.
The players
Cloudflare
Cloudflare is an internet infrastructure and security company offering content delivery, bot management, and serverless computing services to websites and businesses.
Matthew Prince
Matthew Prince is the chief executive of Cloudflare, who has led the company's push into bot management and content protection tools.
Meta
Meta is a large technology company that operates social platforms and is a Cloudflare customer; it is developing new Muse agents that Prince said will increase agentic traffic.
The details
Prince described an AI agent scouring 1,000 restaurant websites and pulling down their menus to choose a single restaurant, illustrating how agentic activity multiplies requests far beyond human browsing. Cloudflare says its products can block agents, reduce the cost and traffic impact of agentic activity, or monetize AI-generated traffic. Prince also said Meta's new Muse agents will generate more agentic traffic that third parties will want to control.
Timeline
July 1: start of the period in which roughly five percent of the web opted into Cloudflare's content protection products.
Thursday: major US indices dropped around four percent.
November 6, 2026: Prince made these remarks on the third-quarter earnings call.
Market Landscape
Cloudflare is positioning its bot-management and content-protection products against AI agent traffic, while also offering serverless Workers as an alternative to containers. The company also said it is among the first to receive quiet disclosure of new vulnerabilities, a position that shapes how it responds to security threats.
Operators whose websites or content are scraped by AI agents should evaluate whether blocking, rate-limiting, or monetizing that traffic fits their cost structure. Teams responsible for bandwidth and security budgets should watch how agentic traffic grows as the bot-to-human ratio shifts.
The takeaway
Agentic traffic can add infrastructure cost without adding revenue, so businesses should measure bot-driven requests against their hosting and security spend. Watch the next quarterly filings for how bot management and content protection revenue develops.
Further reading
Track more company earnings and strategy coverage in the Public Companies section.
Source note: This article includes information reported by TheRegister.
Live Poll
Do you think AI agents crawling websites is a fair cost for businesses to bear?
