prestodb/presto · error · PrestoException

HIVE_INVALID_METADATA

HIVE_INVALID_METADATA

Error message

Table/partition metadata has 'numBuckets' set, but 'bucketCols' is not set

What it means

createBucketProperty validates bucket metadata consistency: if the Glue StorageDescriptor declares numberOfBuckets > 0, bucketColumns must be present. A bucket count without bucket columns is invalid metadata (HIVE_INVALID_METADATA) since the bucketing cannot be applied.

Source

Thrown at presto-hive-metastore/src/main/java/com/facebook/presto/hive/metastore/glue/converter/GlueToPrestoConverter.java:208

        public void setConvertedStorage(StorageDescriptor sd, Storage.Builder storageBuilder)
        {
            requireNonNull(sd.serdeInfo(), "StorageDescriptor SerDeInfo is null");
            SerDeInfo serdeInfo = sd.serdeInfo();

            storageBuilder.setLocation(nullToEmpty(sd.location()))
                    .setBucketProperty(createBucketProperty(sd))
                    .setSkewed(sd.skewedInfo() != null && !isNullOrEmpty(sd.skewedInfo().skewedColumnNames()))
                    .setSerdeParameters(serdeParametersConverter.apply(serdeInfo.parameters()))
                    .setParameters(partitionParametersConverter.apply(sd.parameters()))
                    .setStorageFormat(storageFormatConverter.createStorageFormat(serdeInfo, sd));
        }

        private Optional<HiveBucketProperty> createBucketProperty(StorageDescriptor sd)
        {
            if (sd.numberOfBuckets() > 0) {
                if (isNullOrEmpty(sd.bucketColumns())) {
                    throw new PrestoException(HIVE_INVALID_METADATA, "Table/partition metadata has 'numBuckets' set, but 'bucketCols' is not set");
                }
                List<String> bucketColumns = this.bucketColumns.apply(sd.bucketColumns());
                List<SortingColumn> sortedBy = this.sortColumns.apply(sd.sortColumns());
                return bucketProperty.apply(Optional.of(new HiveBucketProperty(bucketColumns, sd.numberOfBuckets(), sortedBy, HIVE_COMPATIBLE, Optional.empty())));
            }
            return Optional.empty();
        }

        private static List<SortingColumn> createSortingColumns(List<software.amazon.awssdk.services.glue.model.Order> sortColumns)
        {
            if (isNullOrEmpty(sortColumns)) {
                return ImmutableList.of();
            }
            return mappedCopy(sortColumns, column -> new SortingColumn(column.column(), Order.fromMetastoreApiOrder(column.sortOrder(), "unknown")));
        }
    }

    private static final class StorageFormatConverter

View on GitHub (pinned to 55bb57d202)

Solutions

1. Fix the table metadata so numBuckets=0 (unbucketed) or bucketCols is populated
2. Recreate the table with correct CLUSTERED BY (...) INTO n BUCKETS clause
3. Update whatever pipeline wrote incomplete bucketing metadata

Example fix

// before (Glue table)
{"Parameters": {"bucketing_format": ...}, "StorageDescriptor": {"NumberOfBuckets": 16, "BucketColumns": []}}
// after
{"StorageDescriptor": {"NumberOfBuckets": 16, "BucketColumns": ["id"]}}

StorageDescriptor sd = glueTable.storageDescriptor();
if (sd != null && sd.numberOfBuckets() > 0 && (sd.bucketColumns() == null || sd.bucketColumns().isEmpty())) {
    throw new IllegalStateException("Table " + db + "." + t + " has numBuckets but no bucketCols; fix Glue metadata");
}

boolean bucketMetadataValid(StorageDescriptor sd) {
    return sd.numberOfBuckets() == 0 || (sd.bucketColumns() != null && !sd.bucketColumns().isEmpty());
}

try {
    Table table = metastore.getTable(db, tbl);
} catch (PrestoException e) {
    if (e.getErrorCode().getName().equals("HIVE_INVALID_METADATA")) {
        // report inconsistent bucketing metadata
    } else throw e;
}

When it happens

Trigger: Reading a Glue table/partition where sd.numberOfBuckets() > 0 but bucketColumns is null/empty.

Common situations: Manually edited Glue tables, CTAS/ingestion pipelines that set numBuckets without bucketCols, or Hive metadata migration glitches.

Related errors

• unknown java type
• HIVE_INVALID_BUCKET_FILES
• HIVE_INVALID_METADATA
• HIVE_INVALID_METADATA
• Unsupported bucket function type

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/5e257dde4fe1db57. Report an issue: GitHub.