Today, AWS announces the general availability of Neuron 2.24, delivering new features and performance improvements for customers building and deploying deep learning models on AWS Inferentia and Trainium-based instances. Neuron 2.24 introduces support for PyTorch 2.7, enhanced inference capabilities, and expanded compatibility with popular machine learning frameworks. These updates help developers and data scientists accelerate model training and inference, improve efficiency, and simplify the deployment of large language models and other AI workloads.
With Neuron 2.24, customers can take advantage of advanced inference features such as prefix caching for faster Time-To-First-Token (TTFT), disaggregated inference to reduce prefill-decode interference, and context parallelism for improved performance on long sequences. The release also brings support for Qwen 2.5 text models and improved integration with Hugging Face Optimum Neuron and PyTorch-based NxD Core backend.
Neuron 2.24 is available in all AWS Regions where Inferentia and Trainium instances are offered.
To learn more and for a full list of new features and enhancements, see:
Categories: general:products/amazon-machine-learning,marketing:marchitecture/compute,general:products/aws-tools-and-sdks
Source: Amazon Web Services
Latest Posts
- Microsoft 365 Weekly Change Intelligence – 2 Critical & 20 High Impact Changes
- MC1474455: Microsoft Copilot App Adds In-App Browsing on Windows and Mac
- MC1474461: Microsoft 365 Admin Center Adds Granular Browser Access Controls for Copilot Cowork
- MC1474458: Microsoft Copilot Notebooks Adds Power BI Report References for Data-Driven Summaries and Briefs
