Amazon EMR Serverless now offers serverless storage that eliminates local storage provisioning for Apache Spark workloads, reducing data processing costs by up to 20% and preventing job failures from disk capacity constraints. You no longer need to configure local disk type and size for each application. EMR Serverless automatically handles intermediate data operation such as shuffle with no local storage charges. You pay only for compute and memory resources your job consumes.
EMR Serverless offloads intermediate data operations to a fully managed, auto-scaling serverless storage that encrypts data in transit and at rest with job-level isolation. Serverless storage decouples storage from compute, allowing Spark to release workers immediately when idle rather than keeping workers active to preserve temporary data. It eliminates job failures from insufficient disk capacity and reduces costs by avoiding idle worker charges. This is particularly valuable for jobs using dynamic resource allocation, such as recommendation engines processing millions of customer interactions, where initial stages process large datasets with high parallelism then narrow as data aggregates.
This feature is generally available for EMR release 7.12 and later. See Supported AWS Regions for availability. To get started, visit see serverless storage for EMR Serverless documentation.
Categories: general:products/amazon-emr,marketing:marchitecture/analytics,general:products/aws-govcloud-us
Source: Amazon Web Services
Latest Posts
- AWS Lambda Durable Execution SDK for Java GA
- AWS Lambda functions can now mount Amazon S3 buckets as file systems with S3 Files
- (Updated) Microsoft Loop – Require Existing Microsoft 365 Group for New Loop workspaces [MC929022]
- (Updated) Microsoft Teams: Badging updates to help users find messages that count [MC1268911]
