Today, AWS announced that AWS Parallel Computing Service (AWS PCS) supports SPANK (Slurm Plug-in Architecture for Node and job [K]control) plugins, enabling you to extend and modify how Slurm schedules and processes your high performance computing (HPC) workloads without modifying Slurm directly.
Using SPANK plugins, you can now integrate AWS PCS with container technologies, implement custom monitoring of memory and I/O patterns, and dynamically modify job launches to enhance resource management. For example, you can use Enroot and Pyxis plugins to seamlessly run containerized machine learning and HPC workloads using images from Amazon Elastic Container Registry, Docker Hub, NVIDIA NGC, or other container registries.
This feature is available in all AWS Regions where AWS PCS is available. You can implement SPANK plugins by installing and configuring them on the Amazon Machine Images (AMIs) that power your AWS PCS compute nodes. The plugins will be dynamically loaded at runtime when jobs launch, providing a low-effort approach to extend Slurm’s functionality.
To learn more about using SPANK plugins with AWS PCS, see the AWS PCS User Guide.
Categories:
Source: Amazon Web Services
