{"record":{"id":"5e257dde4fe1db57","repo":"prestodb/presto","slug":"hive-invalid-metadata-5e257d","errorCode":"HIVE_INVALID_METADATA","errorMessage":"Table/partition metadata has 'numBuckets' set, but 'bucketCols' is not set","messagePattern":"Table/partition metadata has 'numBuckets' set, but 'bucketCols' is not set","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"error","filePath":"presto-hive-metastore/src/main/java/com/facebook/presto/hive/metastore/glue/converter/GlueToPrestoConverter.java","lineNumber":208,"sourceCode":"\n        public void setConvertedStorage(StorageDescriptor sd, Storage.Builder storageBuilder)\n        {\n            requireNonNull(sd.serdeInfo(), \"StorageDescriptor SerDeInfo is null\");\n            SerDeInfo serdeInfo = sd.serdeInfo();\n\n            storageBuilder.setLocation(nullToEmpty(sd.location()))\n                    .setBucketProperty(createBucketProperty(sd))\n                    .setSkewed(sd.skewedInfo() != null && !isNullOrEmpty(sd.skewedInfo().skewedColumnNames()))\n                    .setSerdeParameters(serdeParametersConverter.apply(serdeInfo.parameters()))\n                    .setParameters(partitionParametersConverter.apply(sd.parameters()))\n                    .setStorageFormat(storageFormatConverter.createStorageFormat(serdeInfo, sd));\n        }\n\n        private Optional<HiveBucketProperty> createBucketProperty(StorageDescriptor sd)\n        {\n            if (sd.numberOfBuckets() > 0) {\n                if (isNullOrEmpty(sd.bucketColumns())) {\n                    throw new PrestoException(HIVE_INVALID_METADATA, \"Table/partition metadata has 'numBuckets' set, but 'bucketCols' is not set\");\n                }\n                List<String> bucketColumns = this.bucketColumns.apply(sd.bucketColumns());\n                List<SortingColumn> sortedBy = this.sortColumns.apply(sd.sortColumns());\n                return bucketProperty.apply(Optional.of(new HiveBucketProperty(bucketColumns, sd.numberOfBuckets(), sortedBy, HIVE_COMPATIBLE, Optional.empty())));\n            }\n            return Optional.empty();\n        }\n\n        private static List<SortingColumn> createSortingColumns(List<software.amazon.awssdk.services.glue.model.Order> sortColumns)\n        {\n            if (isNullOrEmpty(sortColumns)) {\n                return ImmutableList.of();\n            }\n            return mappedCopy(sortColumns, column -> new SortingColumn(column.column(), Order.fromMetastoreApiOrder(column.sortOrder(), \"unknown\")));\n        }\n    }\n\n    private static final class StorageFormatConverter","sourceCodeStart":190,"sourceCodeEnd":226,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-hive-metastore/src/main/java/com/facebook/presto/hive/metastore/glue/converter/GlueToPrestoConverter.java#L190-L226","documentation":"createBucketProperty validates bucket metadata consistency: if the Glue StorageDescriptor declares numberOfBuckets > 0, bucketColumns must be present. A bucket count without bucket columns is invalid metadata (HIVE_INVALID_METADATA) since the bucketing cannot be applied.","triggerScenarios":"Reading a Glue table/partition where sd.numberOfBuckets() > 0 but bucketColumns is null/empty.","commonSituations":"Manually edited Glue tables, CTAS/ingestion pipelines that set numBuckets without bucketCols, or Hive metadata migration glitches.","solutions":["Fix the table metadata so numBuckets=0 (unbucketed) or bucketCols is populated","Recreate the table with correct CLUSTERED BY (...) INTO n BUCKETS clause","Update whatever pipeline wrote incomplete bucketing metadata"],"exampleFix":"// before (Glue table)\n{\"Parameters\": {\"bucketing_format\": ...}, \"StorageDescriptor\": {\"NumberOfBuckets\": 16, \"BucketColumns\": []}}\n// after\n{\"StorageDescriptor\": {\"NumberOfBuckets\": 16, \"BucketColumns\": [\"id\"]}}","handlingStrategy":"validation","validationCode":"StorageDescriptor sd = glueTable.storageDescriptor();\nif (sd != null && sd.numberOfBuckets() > 0 && (sd.bucketColumns() == null || sd.bucketColumns().isEmpty())) {\n    throw new IllegalStateException(\"Table \" + db + \".\" + t + \" has numBuckets but no bucketCols; fix Glue metadata\");\n}","typeGuard":"boolean bucketMetadataValid(StorageDescriptor sd) {\n    return sd.numberOfBuckets() == 0 || (sd.bucketColumns() != null && !sd.bucketColumns().isEmpty());\n}","tryCatchPattern":"try {\n    Table table = metastore.getTable(db, tbl);\n} catch (PrestoException e) {\n    if (e.getErrorCode().getName().equals(\"HIVE_INVALID_METADATA\")) {\n        // report inconsistent bucketing metadata\n    } else throw e;\n}","preventionTips":["Always create bucketed tables with CLUSTERED BY (...) INTO n BUCKETS","Validate ingested table parameters before marking tables readable","Set numBuckets=0 when table is intentionally unbucketed"],"tags":["glue","bucketing","invalid-metadata","hive"],"backgroundTag":"bucket-metadata-missing-bucket-cols","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}