AI Crawl Control

AI Crawl Control now supports per-crawler drilldowns with an extended actions menu and status code analytics. Drill down into Metrics, Cloudflare Radar, and Security Analytics, or export crawler data for use in WAF custom rules, Redirect Rules, and robots.txt files.…

AI Crawl Control now includes a Robots.txt tab that provides insights into how AI crawlers interact with your robots.txt files. What's new The Robots.txt tab allows you to: Monitor the health status of robots.txt files across all your hostnames, including…

AI Crawl Control now provides enhanced metrics and CSV data exports to help you better understand AI crawler activity across your sites. What's new Track crawler requests over time Visualize crawler activity patterns over time, and group data by different…