In this release, AWS Neuron SDK 2.29.0 promotes the Neuron Kernel Interface (NKI) from Beta to Stable with version 0.3.0. NKI gives developers direct, low-level programming access to AWS Trainium and AWS Inferentia NeuronCores using a Python-based syntax. This release introduces the NKI Standard Library, which exposes developer-visible source code for all NKI APIs and native language objects. It also contains a new CPU Simulator that lets developers write, test, and debug NKI kernels locally on standard CPU, without requiring Trainium hardware, using standard Python debugging tools. NKI 0.3.0 also adds new ISA-level features including a dedicated exponential instruction, matmul accumulation control, DMA priority settings for Trn3, and variable-length all-to-all collectives.
The NKI Library expands with 7 new experimental kernels covering Conv1D, a multi-layer Transformer token generation megakernel, fused communication-compute primitives for Trainium2, and dynamic tiling operations. Existing kernels also receive improvements. Attention CTE scales to larger batch sizes and sequence lengths, MLP adds mixed-precision quantization paths, and MoE TKG introduces a dynamic all-expert algorithm.
For inference, NxD Inference improves vision language model support with optimizations for Qwen3 VL and Qwen2 VL, including text-model sequence parallelism and vision data parallelism. vLLM Neuron Plugin updated to version 0.5.0.
Neuron Explorer, Neuron’s profiling and debugging suite of tools, also moves from Beta to Stable. The System Trace Viewer now supports the full set of Device widgets for multi-device profile analysis, and the tool is available on the VS Code Extension Marketplace for streamlined installation. For full release details, see the AWS Neuron SDK 2.29.0 release notes.
The SDK is available in all AWS Regions supporting Inferentia and Trainium instances.
Learn more:
- Neuron Kernel Interface (NKI) Documentation
- vLLM Neuron Plugin Documentation
- Neuron Explorer Documentation
Categories: marketing:marchitecture/artificial-intelligence,marketing:marchitecture/compute
Source: Amazon Web Services
