{"record":{"id":"b936c5bc4922eabd","repo":"prestodb/presto","slug":"hive-missing-data-b936c5","errorCode":"HIVE_MISSING_DATA","errorMessage":"Error opening Hive split %s (offset=%s, length=%s): %s","messagePattern":"Error opening Hive split (.+?) \\(offset=(.+?), length=(.+?)\\): (.+?)","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"critical","filePath":"presto-hive/src/main/java/com/facebook/presto/hive/parquet/ParquetPageSourceFactoryUtils.java","lineNumber":54,"sourceCode":"\n    public static PrestoException mapToPrestoException(Exception e, Path path, HiveFileSplit fileSplit)\n    {\n        if (e instanceof PrestoException) {\n            throw (PrestoException) e;\n        }\n        if (e instanceof ParquetCorruptionException) {\n            throw new PrestoException(HIVE_BAD_DATA, e);\n        }\n        if (e instanceof AccessControlException) {\n            throw new PrestoException(PERMISSION_DENIED, e.getMessage(), e);\n        }\n        if (nullToEmpty(e.getMessage()).trim().equals(\"Filesystem closed\") ||\n                e instanceof FileNotFoundException) {\n            throw new PrestoException(HIVE_CANNOT_OPEN_SPLIT, e);\n        }\n        String message = format(\"Error opening Hive split %s (offset=%s, length=%s): %s\", path, fileSplit.getStart(), fileSplit.getLength(), e.getMessage());\n        if (e.getClass().getSimpleName().equals(\"BlockMissingException\")) {\n            throw new PrestoException(HIVE_MISSING_DATA, message, e);\n        }\n        if (e instanceof HiddenColumnException) {\n            message = format(\"User does not have access to encryption key for encrypted column = %s. If returning 'null' for encrypted \" +\n                    \"columns is acceptable to your query, please add 'set session hive.read_null_masked_parquet_encrypted_value_enabled=true' before your query\", ((HiddenColumnException) e).getColumn());\n            throw new PrestoException(PERMISSION_DENIED, message, e);\n        }\n        throw new PrestoException(HIVE_CANNOT_OPEN_SPLIT, message, e);\n    }\n}\n","sourceCodeStart":36,"sourceCodeEnd":64,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-hive/src/main/java/com/facebook/presto/hive/parquet/ParquetPageSourceFactoryUtils.java#L36-L64","documentation":"mapToPrestoException converts IOExceptions from opening a Hive split's Parquet file into PrestoExceptions. When the underlying exception is a HDFS BlockMissingException (data blocks unavailable on datanodes), it is rethrown as HIVE_MISSING_DATA with a message identifying the split path, offset and length. This signals that the file exists but its data blocks cannot be read from HDFS.","triggerScenarios":"Input stream creation (hdfs.open on the split path) during Parquet page source creation throws a BlockMissingException — i.e., all replicas of a block are missing/failed; split path/start/length are embedded in the message.","commonSituations":"HDFS datanode failures or decommissioned nodes losing block replicas; corrupted/deleted block files on datanode disks; files partially copied to HDFS without replication completing; under-replicated files after datanode crashes.","solutions":["Run 'hdfs fsck <path> -files -blocks -locations' to identify missing blocks and confirm the corruption scope","Restore the missing data: re-copy the source file into HDFS, re-run the upstream write job, or restore from snapshot/backup","Delete the corrupted file/partition and recompute it (e.g. INSERT OVERWRITE or re-run the producing pipeline)","Increase HDFS replication factor and fix datanode health to prevent recurrence; check datanode logs for disk failures"],"exampleFix":"// before\nSELECT * FROM hive.my_table; -- fails: BlockMissingException on /data/part-0001.parquet\n-- after (recreate the bad partition from raw data)\nALTER TABLE my_table DROP IF EXISTS PARTITION (ds='2026-01-01');\nINSERT OVERWRITE TABLE my_table PARTITION (ds='2026-01-01') SELECT ... FROM raw_source WHERE ds='2026-01-01';","handlingStrategy":"retry","validationCode":"// Check block health before scheduling the split read\nhdfs fsck /path/to/part-0001.parquet -files -blocks -locations\n// programmatic:\nDistributedFileSystem dfs = (DistributedFileSystem) FileSystem.get(conf);\nFileStatus st = dfs.getFileStatus(path);\nLocatedBlocks blocks = dfs.getClient().getLocatedBlocks(st.getPath().toString(), 0, st.getLen());\nboolean healthy = blocks.getLocatedBlocks().stream().allMatch(b -> !b.isCorrupt() && b.getLocations().length > 0);","typeGuard":"function isBlockMissingError(e) {\n  return e != null && (e.getClass().getSimpleName() === 'BlockMissingException' ||\n    (e.getCause() != null && e.getCause().getClass().getSimpleName() === 'BlockMissingException'));\n}","tryCatchPattern":"try {\n    readSplit(split);\n} catch (PrestoException e) {\n    if (HIVE_MISSING_DATA.equals(e.getErrorCode())) {\n        markSplitUnhealthy(split); // trigger re-copy of source file or recompute partition\n    } else { throw e; }\n}","preventionTips":["Keep HDFS replication >= 3 and monitor under-replicated/missing-block alerts","Run hdfs fsck regularly on hot table directories and repair before querying","Ensure upstream write jobs complete (no partial files) before making partitions visible","Monitor datanode disk health and replace failing disks promptly"],"tags":["hdfs","parquet","missing-data","datanode"],"backgroundTag":"hdfs-block-missing","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}