AWS Glue now supports read and write operations from AWS Glue 5.0 Apache Spark jobs on AWS Lake Formation registered tables when the job role has full table access. This capability enables Data Manipulation Language (DML) operations including CREATE, ALTER, DELETE, UPDATE, and MERGE INTO statements on Apache Hive and Iceberg tables from within the same Apache Spark application.
While Lake Formation’s fine-grained access control (FGAC) offers granular security controls at row, column, and cell levels, many ETL workloads simply need full table access. This new feature enables AWS Glue 5.0 Spark jobs to directly read and write data when full table access is granted, removing limitations that previously restricted certain Extract, Transform, and Load (ETL) operations. You can now leverage advanced Spark capabilities including Resilient Distributed Datasets (RDDs), custom libraries, and User Defined Functions (UDFs) with Lake Formation tables. Additionally, data teams can run complex, interactive Spark applications through SageMaker Unified Studio in compatibility mode while maintaining Lake Formation’s table-level security boundaries.
This feature is available in all AWS Regions where AWS Glue and AWS Lake Formation are supported. To learn more, visit the AWS Glue product page and documentation.
Categories: marketing:marchitecture/analytics,general:products/aws-glue,general:products/aws-lake-formation
Source: Amazon Web Services
Latest Posts
- Microsoft Teams Rooms on Android: License-based management updates for DM-synced devices [MC1242781]
- Microsoft Purview Insider Risk Management: Pay-as-you-go model for generative AI apps [MC1242784]
- Set a custom name for the OneDrive sync folder [MC1242782]
- Set expiration policies for “People in your organization” sharing links in SharePoint and OneDrive [MC1242772]
![Microsoft PowerPoint for Windows: Create a presentation with Microsoft Copilot from the PowerPoint Backstage view [MC1070846]](https://mwpro.co.uk/wp-content/uploads/2024/08/pexels-singkham-178541-1108572-150x150.webp "Microsoft PowerPoint for Windows: Create a presentation with Microsoft Copilot from the PowerPoint Backstage view [MC1070846] 7")