AI Gateway – Identify model overuse and potential savings with User Insights

AI Gateway User Insights now gives you more context about the traffic flowing through your gateway. It shows what users and agents are doing with AI, and where a selected model may be more capable than a task requires.

On the analysis side, User Insights groups conversations by task, tracks conversation turns, and helps you compare model fit with cost and latency.

User Insights task and model analysis grouped by task categories

The Potential Savings view highlights requests that may work with faster or less expensive models without compromising output quality. These are the same signals that Cloudflare’s Auto Router uses to select a model based on task and cost.

Potential Savings view comparing tasks and suggested models

These new insights are available to all AI Gateway customers at no additional cost. For more information, refer to User Insights.

Source: Cloudflare

Share This Update