AWS Glue 6.0 launches with 30% price cut and full Iceberg v3 support

AWS has announced the general availability of AWS Glue 6.0, a major update that comes with a 30% lower price tag compared to earlier versions and complete support for Apache Iceberg v3 features. The new release runs on a fully modernized runtime built on Apache Spark 4.1, Python 3.12, and Scala 2.13, which delivers faster performance.

Among the standout additions is the VARIANT data type with shredding support, which allows users to store and query semi-structured data like JSON and logs without flattening schemas. This reduces the need for duplicate data copies, custom parsing code, and prevents pipeline failures when schemas shift. AWS claims this is the most complete Iceberg v3 implementation available on any fully serverless managed Spark service.

Other Iceberg v3 capabilities include geometry and geography data types for spatial processing, nanosecond-precision timestamps for high-frequency workloads, and unknown type handling to cope with evolving schemas. The upgrade also brings Spark 4.1 features such as declarative pipelines for simpler ETL authoring, Arrow-native Python UDFs and UDTFs for better PySpark performance, and a real-time streaming mode with single-digit millisecond latency for stateless use cases.

No API changes are required to adopt Glue 6.0. Users can select the new version via the existing --glue-version parameter in the create-job or update-job APIs through the AWS CLI, SDK, Glue Studio, SageMaker Unified Studio, or an IDE. Existing jobs can be migrated using the Spark upgrade agent or the auto-upgrade feature.

AWS Glue 6.0 is now generally available in all AWS Regions where Glue operates. Pricing remains hourly, billed by the second for crawlers and ETL jobs, with the Data Catalog charged a simplified monthly fee - the first million objects stored and accessed are free.

Source: AWS