AWS Glue now supports new workers for larger and memory intensive workloads

AWS Glue now offers new additional worker types to meet diverse data integration and data processing needs. The new workers include larger G.12X and G.16X general compute workers, and four new memory optimized workers, R.1X, R.2X, R.4X, and R.8X, for memory intensive AWS Glue workloads. Glue customers are now able to handle more complex transforms, aggregations, joins, and queries and are able to process higher volumes of data quickly with Apache Spark.

The new G.12X and G.16X workers extend the existing G worker sizes, offering more compute, memory, and storage. These workers are ideal for customers with large and resource-intensive workloads. The new R.1X, R.2X, R.4X, and R.8X workers provide double the memory compared to G workers, making them suitable workloads with memory-intensive Spark operations like caching, shuffling, and aggregating. Customers can select these new worker types in AWS Glue Studio, using notebooks or Visual ETL, or via the Glue Job APIs.

For more information on these new worker types and AWS Regions where the new workers are available, visit the AWS Glue documentation.

Categories:

Source: Amazon Web Services

Latest Posts