{"record":{"id":"062483ef0ff58fea","repo":"prestodb/presto","slug":"hive-invalid-metadata-062483","errorCode":"HIVE_INVALID_METADATA","errorMessage":"Table '%s' or partition '%s' has null columns","messagePattern":"Table '(.+?)' or partition '(.+?)' has null columns","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"error","filePath":"presto-hive/src/main/java/com/facebook/presto/hive/HiveSplitManager.java","lineNumber":633,"sourceCode":"                        if (partitionsNotReadable.size() <= 3) {\n                            partitionsNotReadable.putIfAbsent(reason, new HashSet<>(ImmutableSet.of(partitionName)));\n                            if (partitionsNotReadable.get(reason).size() <= 3) {\n                                partitionsNotReadable.get(reason).add(partitionName);\n                            }\n                        }\n                        continue;\n                    }\n                }\n\n                // Verify that the partition schema matches the table schema.\n                // Either adding or dropping columns from the end of the table\n                // without modifying existing partitions is allowed, but every\n                // column that exists in both the table and partition must have\n                // the same type.\n                List<Column> tableColumns = table.getDataColumns();\n                List<Column> partitionColumns = partition.getColumns();\n                if ((tableColumns == null) || (partitionColumns == null)) {\n                    throw new PrestoException(HIVE_INVALID_METADATA, format(\"Table '%s' or partition '%s' has null columns\", tableName, partitionName));\n                }\n                TableToPartitionMapping tableToPartitionMapping = getTableToPartitionMapping(session, resolvedHiveStorageFormat, tableName, partitionName, tableColumns, partitionColumns);\n\n                if (hiveBucketHandle.isPresent() && !hiveBucketHandle.get().isVirtuallyBucketed()) {\n                    Optional<HiveBucketProperty> partitionBucketProperty = partition.getStorage().getBucketProperty();\n                    if (!partitionBucketProperty.isPresent()) {\n                        throw new PrestoException(HIVE_PARTITION_SCHEMA_MISMATCH, format(\n                                \"Hive table (%s) is bucketed but partition (%s) is not bucketed\",\n                                hivePartition.getTableName(),\n                                hivePartition.getPartitionId().getPartitionName()));\n                    }\n                    int tableBucketCount = hiveBucketHandle.get().getTableBucketCount();\n                    int partitionBucketCount = partitionBucketProperty.get().getBucketCount();\n                    List<String> tableBucketColumns = hiveBucketHandle.get().getColumns().stream()\n                            .map(HiveColumnHandle::getName)\n                            .collect(toImmutableList());\n                    List<String> partitionBucketColumns = partitionBucketProperty.get().getBucketedBy();\n                    if (!tableBucketColumns.equals(partitionBucketColumns) || !isBucketCountCompatible(tableBucketCount, partitionBucketCount)) {","sourceCodeStart":615,"sourceCodeEnd":651,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-hive/src/main/java/com/facebook/presto/hive/HiveSplitManager.java#L615-L651","documentation":"Schema validation during split computation: either the table's dataColumns or the partition's columns list is null in the metastore. Presto requires both column lists to build a TableToPartitionMapping, so it throws HIVE_INVALID_METADATA — the Hive metadata itself is corrupt/incomplete.","triggerScenarios":"computePartitionMetadata reads table.getDataColumns() or partition.getColumns() and gets null, typically because the metastore (or a direct thrift response) has a partition/table row with no SDS/column metadata.","commonSituations":"Partitions created by external tools that wrote incomplete metadata; metastore DB corruption or manual row edits; restore from backup missing column-schema rows.","solutions":["Run MSCK REPAIR TABLE or use ALTER TABLE ADD/DROP COLUMNS to restore column metadata.","Inspect the offending partition's metadata in the metastore (describe formatted <table> partition (...)) and recreate the partition with correct schema.","Recreate the table/partition via Presto or Hive with a proper schema.","Check for external writers or manual metastore edits that produced null columns."],"exampleFix":"// before\nALTER TABLE t ADD IF NOT EXISTS PARTITION (dt='2026-09-01'); -- metadata written with null columns by external tool\n// after\nALTER TABLE t ADD IF NOT EXISTS PARTITION (dt='2026-09-01') LOCATION '...'; -- created via Hive/Presto so column schema is populated","handlingStrategy":"validation","validationCode":"// pre-check via metastore client\nTable t = metastore.getTable(db, name);\nif (t.getSd().getCols() == null) { repairTableMetadata(); }\n// or in SQL first: DESCRIBE <table>; verify columns non-empty","typeGuard":null,"tryCatchPattern":"catch (PrestoException e) { if (e.getErrorCode().getCode() == HIVE_INVALID_METADATA.toErrorCode().getCode()) { /* repair metadata then retry */ } else throw e; }","preventionTips":["Create partitions only through Hive/Presto or writers that populate column schemas.","Run MSCK REPAIR TABLE / schema checks after external ETL writes.","Audit metastore integrity after restores or manual edits."],"tags":["hive","metadata","schema","metastore"],"backgroundTag":"hive-invalid-metadata","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}