Amazon CloudWatch now offers accelerated compute insights, a capability that provides monitoring and observability for self-hosted AI inference workloads running in Amazon EKS clusters. Previously, platform and AI teams lacked fleet-level visibility into their accelerators, whether workloads were running efficiently, and what was causing performance issues. Accelerated compute insights collects telemetry from across the inference stack to help teams identify idle accelerators, optimize workloads to reduce costs, and troubleshoot common inference issues such as high time to first token.
Getting started is as simple as enabling the Amazon CloudWatch Observability add-on for Amazon EKS, which discovers every NVIDIA GPU in the cluster automatically. Curated views in the CloudWatch console let you monitor from the fleet level down to an individual workload or accelerator. Teams can see which workloads are idle, which are underutilized, and the estimated cost per token for each inference workload. The feature is also available on CloudWatch Omni, where it includes an agentic experience that makes answering questions about inference workloads easy.
There is no additional charge for accelerated compute insights. Standard CloudWatch pricing applies to the telemetry you collect from your AI workloads.
For more information about the AWS Regions where Accelerated Compute Insights is available, see the AWS Region table . To get started, see Accelerated Compute Insights .
Categories: general:products/amazon-eks,general:products/amazon-cloudwatch
Source: Amazon Web Services
