prestodb/presto · critical · PrestoException

HIVE_MISSING_DATA

HIVE_MISSING_DATA

Error message

Error opening Hive split %s (offset=%s, length=%s): %s

What it means

mapToPrestoException converts IOExceptions from opening a Hive split's Parquet file into PrestoExceptions. When the underlying exception is a HDFS BlockMissingException (data blocks unavailable on datanodes), it is rethrown as HIVE_MISSING_DATA with a message identifying the split path, offset and length. This signals that the file exists but its data blocks cannot be read from HDFS.

Source

Thrown at presto-hive/src/main/java/com/facebook/presto/hive/parquet/ParquetPageSourceFactoryUtils.java:54

    public static PrestoException mapToPrestoException(Exception e, Path path, HiveFileSplit fileSplit)
    {
        if (e instanceof PrestoException) {
            throw (PrestoException) e;
        }
        if (e instanceof ParquetCorruptionException) {
            throw new PrestoException(HIVE_BAD_DATA, e);
        }
        if (e instanceof AccessControlException) {
            throw new PrestoException(PERMISSION_DENIED, e.getMessage(), e);
        }
        if (nullToEmpty(e.getMessage()).trim().equals("Filesystem closed") ||
                e instanceof FileNotFoundException) {
            throw new PrestoException(HIVE_CANNOT_OPEN_SPLIT, e);
        }
        String message = format("Error opening Hive split %s (offset=%s, length=%s): %s", path, fileSplit.getStart(), fileSplit.getLength(), e.getMessage());
        if (e.getClass().getSimpleName().equals("BlockMissingException")) {
            throw new PrestoException(HIVE_MISSING_DATA, message, e);
        }
        if (e instanceof HiddenColumnException) {
            message = format("User does not have access to encryption key for encrypted column = %s. If returning 'null' for encrypted " +
                    "columns is acceptable to your query, please add 'set session hive.read_null_masked_parquet_encrypted_value_enabled=true' before your query", ((HiddenColumnException) e).getColumn());
            throw new PrestoException(PERMISSION_DENIED, message, e);
        }
        throw new PrestoException(HIVE_CANNOT_OPEN_SPLIT, message, e);
    }
}

View on GitHub (pinned to 55bb57d202)

Solutions

1. Run 'hdfs fsck <path> -files -blocks -locations' to identify missing blocks and confirm the corruption scope
2. Restore the missing data: re-copy the source file into HDFS, re-run the upstream write job, or restore from snapshot/backup
3. Delete the corrupted file/partition and recompute it (e.g. INSERT OVERWRITE or re-run the producing pipeline)
4. Increase HDFS replication factor and fix datanode health to prevent recurrence; check datanode logs for disk failures

Example fix

// before
SELECT * FROM hive.my_table; -- fails: BlockMissingException on /data/part-0001.parquet
-- after (recreate the bad partition from raw data)
ALTER TABLE my_table DROP IF EXISTS PARTITION (ds='2026-01-01');
INSERT OVERWRITE TABLE my_table PARTITION (ds='2026-01-01') SELECT ... FROM raw_source WHERE ds='2026-01-01';

// Check block health before scheduling the split read
hdfs fsck /path/to/part-0001.parquet -files -blocks -locations
// programmatic:
DistributedFileSystem dfs = (DistributedFileSystem) FileSystem.get(conf);
FileStatus st = dfs.getFileStatus(path);
LocatedBlocks blocks = dfs.getClient().getLocatedBlocks(st.getPath().toString(), 0, st.getLen());
boolean healthy = blocks.getLocatedBlocks().stream().allMatch(b -> !b.isCorrupt() && b.getLocations().length > 0);

function isBlockMissingError(e) {
  return e != null && (e.getClass().getSimpleName() === 'BlockMissingException' ||
    (e.getCause() != null && e.getCause().getClass().getSimpleName() === 'BlockMissingException'));
}

try {
    readSplit(split);
} catch (PrestoException e) {
    if (HIVE_MISSING_DATA.equals(e.getErrorCode())) {
        markSplitUnhealthy(split); // trigger re-copy of source file or recompute partition
    } else { throw e; }
}

When it happens

Trigger: Input stream creation (hdfs.open on the split path) during Parquet page source creation throws a BlockMissingException — i.e., all replicas of a block are missing/failed; split path/start/length are embedded in the message.

Common situations: HDFS datanode failures or decommissioned nodes losing block replicas; corrupted/deleted block files on datanode disks; files partially copied to HDFS without replication completing; under-replicated files after datanode crashes.

Related errors

• HIVE_MISSING_DATA
• ICEBERG_MISSING_DATA
• PARQUET_IO_READ_ERROR
• PARQUET_IO_READ_ERROR
• Dictionary is missing for Page

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/b936c5bc4922eabd. Report an issue: GitHub.