prestodb/presto · error · PrestoException

HIVE_MISSING_DATA

HIVE_MISSING_DATA

Error message

Error opening Hive split %s (offset=%s, length=%s): %s

What it means

RcFilePageSourceFactory detects Hadoop BlockMissingException while constructing the RCFile reader and rethrows it as PrestoException HIVE_MISSING_DATA, with the message 'Error opening Hive split <path> (offset=..., length=...): ...'. It means HDFS reported that a block needed by the split is not available on any DataNode — data exists in the namespace but is unreadable.

Source

Thrown at presto-hive/src/main/java/com/facebook/presto/hive/rcfile/RcFilePageSourceFactory.java:171

                    new DataSize(8, Unit.MEGABYTE));

            return Optional.of(new RcFilePageSource(rcFileReader, columns, typeManager));
        }
        catch (Throwable e) {
            try {
                inputStream.close();
            }
            catch (IOException ignored) {
            }
            if (e instanceof PrestoException) {
                throw (PrestoException) e;
            }
            String message = splitError(e, fileSplit);
            if (e instanceof RcFileCorruptionException) {
                throw new PrestoException(HIVE_BAD_DATA, message, e);
            }
            if (e.getClass().getSimpleName().equals("BlockMissingException")) {
                throw new PrestoException(HIVE_MISSING_DATA, message, e);
            }
            throw new PrestoException(HIVE_CANNOT_OPEN_SPLIT, message, e);
        }
    }

    private static String splitError(Throwable t, HiveFileSplit fileSplit)
    {
        return format("Error opening Hive split %s (offset=%s, length=%s): %s", fileSplit.getPath(), fileSplit.getStart(), fileSplit.getLength(), t.getMessage());
    }

    public static TextRcFileEncoding createTextVectorEncoding(Properties schema, DateTimeZone hiveStorageTimeZone)
    {
        // separators
        int nestingLevels;
        if (!"true".equalsIgnoreCase(schema.getProperty(SERIALIZATION_EXTEND_NESTING_LEVELS))) {
            nestingLevels = TEXT_LEGACY_NESTING_LEVELS;
        }
        else {

View on GitHub (pinned to 55bb57d202)

Solutions

1. Run `hdfs fsck <path> -files -blocks -locations` to identify the missing blocks.
2. Restore replication: check DataNode health, restart failed DataNodes, or `hdfs debug recoverLease -path <file> -retries 3`.
3. Restore the file from backup/snapshot or re-run the producing job for the affected partition.
4. Avoid decommissioning/balancer operations while queries scan the affected data.
5. Drop the affected partition if the data is unrecoverable.

Example fix

// before: fsck shows MISSING blocks for the split's file
$ hdfs fsck /data/rc/part=1/file.rcfile
// after: restore replication from backup
hdfs dfs -rm /data/rc/part=1/file.rcfile;
hdfs dfs -put /backup/file.rcfile /data/rc/part=1/;
hdfs dfs -setrep -w 3 /data/rc/part=1/file.rcfile;

// check block health before scheduling heavy scans
// hdfs fsck <table-path> -files -blocks -locations | grep -i MISSING
boolean hasMissingBlocks(String path) throws IOException {
    Process p = new ProcessBuilder("hdfs", "fsck", path, "-files", "-blocks", "-locations").start();
    String out = new String(p.getInputStream().readAllBytes());
    return out.toUpperCase().contains("MISSING");
}

boolean isHiveMissingData(Throwable t) {
    return t instanceof PrestoException && ((PrestoException) t).getErrorCode().getName().equals("HIVE_MISSING_DATA");
}

try {
    return query(sql);
} catch (PrestoException e) {
    if (e.getErrorCode().getName().equals("HIVE_MISSING_DATA")) {
        // missing blocks rarely self-heal; restore replication first, then retry once
        recoverLeaseAndReplicate(extractPath(e.getMessage()));
        return query(sql);
    }
    throw e;
}

When it happens

Trigger: openFile succeeds but RcFileReader/HdfsRcFileDataSource reads hit a block with no live replicas: DataNodes lost disks, under-replicated blocks after DataNode decommission, blocks deleted by balancer/hdfs fsck issues, or replica corruption marks the block unavailable.

Common situations: DataNode failure or disk replacement; hdfs fsck reports MISSING blocks; query after a NameNode restart with stale block reports; under-replicated files after decommissioning nodes too quickly.

Related errors

• HIVE_BAD_DATA
• HIVE_CURSOR_ERROR
• HIVE_BAD_DATA
• HIVE_CANNOT_OPEN_SPLIT
• HIVE_FILESYSTEM_ERROR

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/849eeb67c2f8deed. Report an issue: GitHub.