{"record":{"id":"4765a759146c1504","repo":"prestodb/presto","slug":"hive-partition-schema-mismatch-4765a7","errorCode":"HIVE_PARTITION_SCHEMA_MISMATCH","errorMessage":"The column %s of table %s is declared as type %s, but the Parquet file (%s) declares the column as type %s","messagePattern":"The column (.+?) of table (.+?) is declared as type (.+?), but the Parquet file \\((.+?)\\) declares the column as type (.+?)","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"error","filePath":"presto-hive/src/main/java/com/facebook/presto/hive/parquet/ParquetPageSourceFactory.java","lineNumber":391,"sourceCode":"            type = messageType.getType(column.getHiveColumnIndex());\n        }\n\n        if (type == null) {\n            return Optional.empty();\n        }\n\n        if (!checkSchemaMatch(type, prestoType)) {\n            String parquetTypeName;\n            if (type.isPrimitive()) {\n                parquetTypeName = type.asPrimitiveType().getPrimitiveTypeName().toString();\n            }\n            else {\n                GroupType group = type.asGroupType();\n                StringBuilder builder = new StringBuilder();\n                group.writeToStringBuilder(builder, \"\");\n                parquetTypeName = builder.toString();\n            }\n            throw new PrestoException(HIVE_PARTITION_SCHEMA_MISMATCH, format(\"The column %s of table %s is declared as type %s, but the Parquet file (%s) declares the column as type %s\",\n                    column.getName(),\n                    tableName.toString(),\n                    column.getHiveType(),\n                    path.toString(),\n                    parquetTypeName));\n        }\n        return Optional.of(type);\n    }\n\n    public static boolean checkSchemaMatch(org.apache.parquet.schema.Type parquetType, Type type)\n    {\n        String prestoType = type.getTypeSignature().getBase();\n        if (parquetType instanceof GroupType) {\n            GroupType groupType = parquetType.asGroupType();\n            switch (prestoType) {\n                case ROW:\n                    RowType rowType = (RowType) type;\n                    Map<String, Type> prestoFieldMap = rowType.getFields().stream().collect(","sourceCodeStart":373,"sourceCodeEnd":409,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-hive/src/main/java/com/facebook/presto/hive/parquet/ParquetPageSourceFactory.java#L373-L409","documentation":"Presto throws HIVE_PARTITION_SCHEMA_MISMATCH from getParquetType when a column's Hive-declared type differs from the type found in the Parquet file's schema. The Hive metastore schema and the physical file are compared per column; any divergence (including nested group types, whose Parquet name is built via writeToStringBuilder) causes the read to be aborted rather than silently returning wrong data.","triggerScenarios":"Reading a Parquet-backed Hive table where column.getName()'s Hive type (column.getHiveType()) does not match the Parquet schema's type for that column; typically hit inside createParquetPageSource when a split is read after the table schema was altered without rewriting files, or when files written by a different writer (different Parquet types, e.g. INT96 vs TIMESTAMP, INT32 vs INT64) were registered under the table.","commonSituations":"ALTER TABLE CHANGE COLUMN / Hive schema evolution on data not rewritten; writing files with Spark/Impala using incompatible logical types then querying via Presto; partition-level schema drift where one partition's files differ from the metastore schema; timestamp columns stored as INT96 vs annotated TIMESTAMP_MICROS.","solutions":["Align the metastore schema with the actual Parquet file types (ALTER TABLE ... CHANGE COLUMN back, or recreate the table with the file's types)","Rewrite the mismatched files so their Parquet types match the declared Hive schema (INSERT OVERWRITE / CTAS)","If a specific partition's files are wrong, rewrite only that partition's files or drop/replace the partition","Check which writer produced the files and configure it to emit Parquet types matching the Hive schema (e.g. disable INT96 timestamps, use spark.sql.parquet.int96TimestampConversion appropriately)"],"exampleFix":"// before: files written as INT32 while table declares bigint\nCREATE TABLE t (id int) STORED AS PARQUET; -- files have int32 but metastore says bigint after ALTER\n-- after: rewrite files to match schema\nINSERT OVERWRITE TABLE t SELECT CAST(id AS INT) AS id FROM t_raw; -- or fix metastore to match files","handlingStrategy":"validation","validationCode":"// Validate Parquet file schema against the metastore table before querying/ingesting\nParquetFileReader reader = ParquetFileReader.open(conf, new Path(filePath));\nMessageType fileSchema = reader.getFileMetaData().getSchema();\nfor (ColumnHandle handle : expectedColumns) {\n    HiveColumnHandle col = (HiveColumnHandle) handle;\n    Type fileCol = fileSchema.getType(col.getName());\n    if (fileCol == null || !parquetTypeMatches(col.getHiveType(), fileCol)) {\n        throw new IllegalStateException(\"Schema mismatch for column \" + col.getName() +\n            \": metastore=\" + col.getHiveType() + \" file=\" + fileCol);\n    }\n}","typeGuard":"function parquetTypeMatches(hiveType, parquetType) {\n  if (parquetType == null) return false;\n  const normalized = normalizeHiveType(hiveType); // e.g. 'bigint'->'int64'\n  return normalized === parquetType.toString();\n}","tryCatchPattern":"try {\n    readParquetSplit(split);\n} catch (PrestoException e) {\n    if (HIVE_PARTITION_SCHEMA_MISMATCH.equals(e.getErrorCode())) {\n        LOG.error(\"Falling back: file %s does not match table schema\", split.getPath());\n        // route to repair/requeue rather than failing the whole query\n    } else { throw e; }\n}","preventionTips":["After ALTER TABLE CHANGE COLUMN, always rewrite affected files (INSERT OVERWRITE) so physical types match","Standardize one writer (version + settings) for Parquet files registered in the same table","Run a periodic schema-consistency check (metastore vs parquet footer) over partitions","Avoid INT96 timestamps; declare timestamps in the table exactly as written (TIMESTAMP_MICROS etc.)"],"tags":["hive","parquet","schema-mismatch","metastore"],"backgroundTag":"schema-mismatch-between-metastore-and-parquet-file","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}