Amazon Redshift now supports Just-In-Time (JIT) ANALYZE for Apache Iceberg tables

Amazon Redshift today announces the general availability of Just-In-Time (JIT) ANALYZE capability for Apache Iceberg tables, enabling users to run high performance read and write analytics queries on Apache Iceberg tables within the Redshift data lake. The Apache Iceberg open table format has been used by many customers to simplify data processing on rapidly expanding and evolving tables stored in data lakes.

Unlike traditional data warehouses, data lakes often lack comprehensive table-level and column-level statistics about the underlying data, making it challenging for query engines to choose the most optimal query execution plans without visibility into the table and column statistics. Sub-optimal query execution plans can lead to slower and less predictable performance.

‘JIT ANALYZE’ is a new Amazon Redshift feature that automatically collects and utilizes statistics for Iceberg tables during query execution, eliminating manual statistics collection while giving the query engine the information it needs to generate optimal query execution plans. The system uses intelligent heuristics to identify queries that will benefit from statistics, maintains lightweight sketch data structures, and builds high quality table-level and column-level statistics. JIT ANALYZE delivers out-of-the-box performance on par with queries that have pre-calculated statistics, while providing the foundation for many other performance optimizations.

The Amazon Redshift JIT ANALYZE feature for Apache Iceberg tables is now available in all AWS regions where Amazon Redshift is available. Users do not need to make any changes or enable any settings to take advantage of this new data lake query optimization. To get started, visit the documentation page for Amazon Redshift Management Guide.

Categories: marketing:marchitecture/analytics,general:products/amazon-redshift

Source: Amazon Web Services

Latest Posts