AI bot & crawler traffic

Copy link

AI bots scan public websites to collect data used in search engines, AI model training, and other data processing tasks


HTTP traffic by bot

Copy link

HTTP request trends for top five most active AI bots

Learn more...View in Data Explorer

Crawl purpose

Copy link

Share of traffic by the purpose of the crawl

Learn more...View in Data Explorer

Crawl-to-refer ratio

Copy link

Ratio of HTML page crawl requests to HTML page referrals by platform. Change reflects comparison with the previous period

Learn more...View in Data Explorer

AI bot best practices

Copy link

Collective behavior observed by Cloudflare from bots of leading AI models and service providers, sorted by traffic volume

Learn more...

Latest AI blog posts

Copy link

Posts tagged with "AI" from the Cloudflare blog