prestodb/presto · error · OrcCorruptionException

Value is not null but scale stream is not present

Error message

Value is not null but scale stream is not present

What it means

DecimalBatchStreamReader.readBlock throws OrcCorruptionException when the present stream is null (all rows non-null) but the SECONDARY scale stream for the DECIMAL column is missing. ORC decimal encoding requires both a data stream for unscaled values and a secondary stream for scales, so a missing scale stream makes decoding impossible.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/reader/DecimalBatchStreamReader.java:116

        seekToOffset();

        if (decimalStream == null && scaleStream == null && presentStream != null) {
            presentStream.skip(nextBatchSize);
            Block nullValueBlock = RunLengthEncodedBlock.create(type, null, nextBatchSize);
            readOffset = 0;
            nextBatchSize = 0;
            return nullValueBlock;
        }

        BlockBuilder builder = type.createBlockBuilder(null, nextBatchSize);

        if (presentStream == null) {
            if (decimalStream == null) {
                throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), "Value is not null but decimal stream is not present");
            }
            if (scaleStream == null) {
                throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), "Value is not null but scale stream is not present");
            }

            for (int i = 0; i < nextBatchSize; i++) {
                long sourceScale = scaleStream.next();
                if (type.isShort()) {
                    long rescaledDecimal = Decimals.rescale(decimalStream.nextLong(), (int) sourceScale, type.getScale());
                    type.writeLong(builder, rescaledDecimal);
                }
                else {
                    Slice decimal = UnscaledDecimal128Arithmetic.unscaledDecimal();
                    Slice rescaledDecimal = UnscaledDecimal128Arithmetic.unscaledDecimal();

                    decimalStream.nextLongDecimal(decimal);
                    rescale(decimal, (int) (type.getScale() - sourceScale), rescaledDecimal);
                    type.writeSlice(builder, rescaledDecimal);
                }
            }
        }

View on GitHub (pinned to 55bb57d202)

Solutions

1. Regenerate the ORC file with a spec-compliant writer that emits both decimal and scale streams.
2. Diagnose with orc-tools meta/scan and upgrade the producing writer if buggy.
3. Verify complete file transfer/replication (no partial copies).
4. Catch OrcCorruptionException and handle per policy (skip split, surface table error).

Example fix

// before
BigDecimal v = readDecimalColumn(); // throws on missing scale stream
// after
try { v = readDecimalColumn(); } catch (OrcCorruptionException e) { v = null; /* regenerate file */ }

// check for SECONDARY scale stream on decimal columns: orc-tools meta file.orc

try { block = streamReader.readBlock(); } catch (OrcCorruptionException e) { handleCorruptSplit(e); }

When it happens

Trigger: readBlock on a DECIMAL column where presentStream == null and scaleStream == null — the stripe lacks the scale (secondary) stream though decimal values are expected.

Common situations: Files written by writers that omit secondary streams (non-conformant or buggy versions); truncated uploads; stripe metadata corruption mislabeling streams.

Related errors

• Value is not null but decimal stream is not present
• Value is not null but data stream is not present
• Value is not null but data stream is missing
• Value is null but present stream is missing
• Value is null but present stream is missing

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/935cc4d6bf763262. Report an issue: GitHub.