prestodb/presto · error · OrcCorruptionException

EOF while reading unsigned vint

Error message

EOF while reading unsigned vint

What it means

Thrown by LongDecode.readUnsignedVInt when the underlying input reaches EOF (-1) while consuming an unsigned varint. ORC encodes many integers (lengths, dictionaries, secondary streams) as varints; a varint that never terminates before end-of-stream means the stream is truncated or corrupt, so the reader aborts instead of returning a bogus value.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/stream/LongDecode.java:138

    }

    public static long readSignedVInt(OrcInputStream inputStream)
            throws IOException
    {
        long result = readUnsignedVInt(inputStream);
        return zigzagDecode(result);
    }

    private static long readUnsignedVInt(OrcInputStream inputStream)
            throws IOException
    {
        long result = 0;
        int offset = 0;
        long b;
        do {
            b = inputStream.read();
            if (b == -1) {
                throw new OrcCorruptionException(inputStream.getOrcDataSourceId(), "EOF while reading unsigned vint");
            }
            result |= (b & 0b0111_1111) << offset;
            offset += 7;
        }
        while ((b & 0b1000_0000) != 0);
        return result;
    }

    public static long readVInt(boolean signed, OrcInputStream inputStream)
            throws IOException
    {
        if (signed) {
            return readSignedVInt(inputStream);
        }
        else {
            return readUnsignedVInt(inputStream);
        }
    }

View on GitHub (pinned to 55bb57d202)

Solutions

1. Verify the file with orc-tools meta/orc-dump; a varint EOF confirms truncation — re-copy from the source of truth.
2. Regenerate the ORC file from source data.
3. Enable/verify storage checksums to catch corruption during transfer or at rest.
4. Upgrade the writer (Hive/Spark/Presto) if its ORC writer version is known to emit incorrect stream lengths.
5. Wait for writing jobs to commit before querying external tables pointing at in-progress files.

// pre-read integrity check
try (InputStream in = open(path)) {
    byte[] ps = OrcMagic.MAGIC.getBytes(StandardCharsets.US_ASCII);
    byte[] head = in.readNBytes(ps.length);
    if (!Arrays.equals(head, ps)) { throw new IllegalStateException("not a complete ORC file: " + path); }
}

try {
    long v = LongDecode.readVInt(inputStream);
} catch (OrcCorruptionException e) {
    if (e.getMessage().contains("EOF while reading unsigned vint")) {
        return recovery.recopyFileAndReopen(path);
    }
    throw e;
}

When it happens

Trigger: readUnsignedVInt (and readVInt built on it) loops while the high bit is set; input.read() returns -1 before a byte with the high bit clear is seen.

Common situations: ORC files truncated during upload/download; interrupted HDFS writes without checksums; corrupted S3 objects (bad multipart copy); footer declaring more values than the stream actually contains from a buggy writer.

Related errors

• Read past end of buffer RLE byte
• Reading RLE byte got EOF
• Unexpected end of stream
• Reading BigInteger past EOF
• Read past end of RLE integer

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/ae77e91c55224246. Report an issue: GitHub.