Amazon S3 Tables Adds Managed Migration from Apache Hive to Apache Iceberg

Amazon S3 Tables now supports managed migration of Apache Hive tables to Apache Iceberg, giving teams a simpler, lower-cost route to modernizing their data lakes. The feature works with Parquet tables in Amazon S3 registered in the AWS Glue Data Catalog, and requires no pipelines to build or compute to provision while production workloads continue running.

Apache Iceberg has become the open standard for large analytics datasets, supporting petabyte-scale tables with schema evolution, hidden partitioning and time travel queries while remaining queryable by any Iceberg-compatible engine. S3 Tables provide fully managed storage built to keep those tables performant and cost-effective as they grow, with automatic maintenance, optimization, replication and S3 Intelligent-Tiering included.

In the past, moving Hive tables to Iceberg could take weeks of engineering work to build and manage the necessary infrastructure, with data type conversions and rollback capabilities adding operational complexity. S3 Tables now handle the whole workflow as a managed, API-driven operation, covering data validation, compatibility checks, type conversions, file recompression and compaction. Before copying any data, the service identifies every source object in scope and produces a compatibility report. Users can sync as often as needed without stopping writes to the source table, apart from a few minutes during cutover, keeping downtime to a minimum.

Pricing is based only on data copied, with no charge for the Hive-to-Iceberg conversion, and source objects in S3 Intelligent-Tiering stay in their current access tier. To begin, users pick a destination table bucket in the S3 console and choose Create table from migration, then set a namespace and table name; the destination is created with recommended defaults tuned for query performance and storage efficiency. The destination table stays read-only during migration and is promoted to read-write once the process completes.

Data can be queried right away through direct integration with Amazon Athena and Amazon SageMaker Unified Studio in the AWS console, or with any tool compatible with the Apache Iceberg REST API. Migration jobs can also be set up through the AWS CLI, AWS SDKs or the Agent Toolkit for AWS, including the start-table-migration and sync-table-migration commands. A migrating-hive-tables-to-s3-tables skill in the Agent Toolkit offers step-by-step guidance with safety guardrails for coding agents such as Kiro.

The capability is available in US East (N. Virginia, Ohio), US West (Oregon), Asia Pacific (Mumbai, Singapore, Sydney, Tokyo) and Europe (Frankfurt, Ireland, Stockholm). At launch it supports AWS Glue-registered Hive tables stored in Parquet format within the same AWS account and Region.

Source: AWS