Amazon CloudWatch now enables automated quality assessment of AI agents through AgentCore Evaluations. This new capability helps developers continuously monitor and improve agent performance based on real-world interactions, allowing teams to identify and address quality issues before they impact customers.
AgentCore Evaluations comes with 13 pre-built evaluators covering essential quality dimensions like helpfulness, tool selection, and response accuracy, while also supporting custom model-based scoring systems. You can access unified quality metrics and agent telemetry in CloudWatch dashboards, with end-to-end tracing capabilities to correlate evaluation metrics with prompts and logs. The feature integrates seamlessly with CloudWatch’s existing capabilities including Application Signals, Alarms, Sensitive Data Protection, and Logs Insights. This capability eliminates the need for teams to build and maintain custom evaluation infrastructure, accelerating the deployment of high-quality AI agents. Developers can monitor their entire agent fleet through the AgentCore section in the CloudWatch GenAI observability console.
AgentCore Evaluations is now available in US East (N. Virginia), US West (Oregon), Europe (Frankfurt), and Asia Pacific (Sydney). To get started, visit the documentation and pricing details. Standard CloudWatch pricing applies for underlying telemetry data.
Categories: marketing:marchitecture/management-and-governance,general:products/amazon-cloudwatch
Source: Amazon Web Services
Latest Posts
- Amazon SageMaker Feature Store now supports SageMaker Python SDK V3
- Amazon Route 53 Domains adds support for 34 new Top Level Domains including .app, .dev, and .health.
- Amazon EventBridge Scheduler adds 619 new SDK API actions, including Lambda Managed Instances
- (Updated) Anthropic Claude is now available in Microsoft 365 Copilot [MC1247880]
![Flexible layout for Teams meetings with resizable divider [MC1190199]](https://mwpro.co.uk/wp-content/uploads/2025/06/pexels-ashamplifies-3663024-96x96.webp "Flexible layout for Teams meetings with resizable divider [MC1190199] 7")