AWS announces support for NVIDIA Inference Xfer Library (NIXL) with Elastic Fabric Adapter (EFA) to accelerate disaggregated large language model (LLM) inference on Amazon EC2. This integration enhances disaggregated inference serving through three key improvements: increased KV-cache throughput, reduced inter-token latency, and optimized KV-cache memory utilization.
NIXL with EFA enables high throughput, low-latency KV-cache transfer between prefill and decode nodes, and it enables efficient KV-cache movement between various storage layers. NIXL is interoperable with all EFA-enabled EC2 instances and integrates natively with frameworks including NVIDIA Dynamo, SGLang, and vLLM. Combined, NIXL with EFA enables flexible integration with your EC2 instance and framework of choice, providing performant disaggregated inference at scale.
AWS supports NIXL version 1.0.0 or higher with EFA installer version 1.47.0 or higher on all EFA-enabled EC2 instance types in all AWS regions at no additional cost. For more information, visit the EFA documentation.
Categories: marketing:marchitecture/networking-and-content-delivery
Source: Amazon Web Services
Latest Posts
- MC1528011: Microsoft Teams Adds External Presenter Access to Event Group Chat

- MC1519871: Microsoft Copilot Code Expands to Mobile and Mac and Adds Local Sandbox for Frontier Users

- MC1488488: Microsoft Dataverse Adds Bulk AI Generation for Existing Prompt Column Values

- MC1486291: Microsoft 365 Copilot Adds Admin-Led Guided Onboarding Experience for Frontier Users







