prestodb/presto · error · PrestoException

HIVE_INVALID_METADATA

HIVE_INVALID_METADATA

Error message

Table '%s.%s' is bucketed on non-existent column '%s'

What it means

getHiveBucketHandle resolves a table's bucketing property (HiveBucketProperty) into actual HiveColumnHandles by looking up each bucketed-by column name in the table's column map. If the metastore says the table is bucketed on a column name that does not exist among the table's columns, this PrestoException (HIVE_INVALID_METADATA) is thrown because Presto cannot compute the bucketing function without a valid column.

Source

Thrown at presto-hive/src/main/java/com/facebook/presto/hive/HiveBucketing.java:309

    public static Optional<HiveBucketHandle> getHiveBucketHandle(ConnectorSession session, Table table)
    {
        Optional<HiveBucketProperty> hiveBucketProperty = table.getStorage().getBucketProperty();
        if (!hiveBucketProperty.isPresent()) {
            if (table.getTableType().equals(TEMPORARY_TABLE)) {
                return Optional.of(HiveBucketHandle.createVirtualBucketHandle(getCteVirtualBucketCount(session)));
            }
            return Optional.empty();
        }

        Map<String, HiveColumnHandle> map = getRegularColumnHandles(table).stream()
                .collect(Collectors.toMap(HiveColumnHandle::getName, identity()));

        ImmutableList.Builder<HiveColumnHandle> bucketColumns = ImmutableList.builder();
        for (String bucketColumnName : hiveBucketProperty.get().getBucketedBy()) {
            HiveColumnHandle bucketColumnHandle = map.get(bucketColumnName);
            if (bucketColumnHandle == null) {
                throw new PrestoException(
                        HIVE_INVALID_METADATA,
                        format("Table '%s.%s' is bucketed on non-existent column '%s'", table.getDatabaseName(), table.getTableName(), bucketColumnName));
            }
            bucketColumns.add(bucketColumnHandle);
        }

        int bucketCount = hiveBucketProperty.get().getBucketCount();
        return Optional.of(new HiveBucketHandle(bucketColumns.build(), bucketCount, bucketCount));
    }

    public static Optional<HiveBucketFilter> getHiveBucketFilter(Table table, TupleDomain<ColumnHandle> effectivePredicate, boolean useLegacyTimestampBucketing)
    {
        return getHiveBucketFilter(table.getStorage().getBucketProperty(), table.getDataColumns(), effectivePredicate, useLegacyTimestampBucketing);
    }

    public static Optional<HiveBucketFilter> getHiveBucketFilter(
            Optional<HiveBucketProperty> hiveBucketProperty,
            List<Column> dataColumns,

View on GitHub (pinned to 55bb57d202)

Solutions

1. Inspect SHOW CREATE TABLE and compare the bucketed-by names in TBLPROPERTIES ('bucketing_format' / CLUSTERED BY) with the actual column list; correct the mismatch.
2. Restore the dropped/renamed bucket column, or drop and recreate the table's bucketing spec so it references existing columns.
3. Fix the metastore metadata directly (or re-create the table with the correct CLUSTERED BY clause) so the bucketing spec matches the schema.
4. Recreate the table with CTAS and a valid bucketing clause if metadata repair is not feasible.

Example fix

-- before: bucketing spec references a dropped column
CREATE TABLE t (a INT, b STRING) CLUSTERED BY (old_col) INTO 8 BUCKETS;

-- after: bucketing spec references an existing column
CREATE TABLE t (a INT, b STRING) CLUSTERED BY (a) INTO 8 BUCKETS;

-- before querying: ensure bucketed-by columns are present in the schema
SHOW CREATE TABLE my_table;
-- confirm every column named in CLUSTERED BY / bucketing_format still exists;
-- if any was dropped or renamed, repair the bucketing spec or restore the column.

try { ResultSet rs = stmt.executeQuery("SELECT * FROM my_table"); ... }
catch (SQLException e) {
  if (e.getMessage() != null && e.getMessage().contains("is bucketed on non-existent column")) {
    // repair metastore bucketing metadata or recreate the table
  } else throw e;
}

When it happens

Trigger: Reading or querying metadata of a Hive table whose serde/table properties declare CLUSTERED BY (bucketed by) a column name absent from the table's column list — e.g. the column was dropped or renamed after the table was bucketed, or the metastore metadata is stale/corrupt.

Common situations: ALTER TABLE ... DROP COLUMN or RENAME COLUMN on a bucketed table done outside Presto (via Hive CLI) leaving the bucketing spec stale; manually edited metastore/CMS properties; migrating tables between metastores where schema changes were not propagated; tables created with typos in the CLUSTERED BY clause.

Related errors

• HIVE_INVALID_METADATA
• HIVE_INVALID_METADATA
• ARROW_FLIGHT_METADATA_ERROR
• ARROW_FLIGHT_METADATA_ERROR
• GENERIC_INTERNAL_ERROR

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/280f3aad45c70692. Report an issue: GitHub.