prestodb/presto · error · RedisProviderSerdeException

Error decoding historicalPlanStatistics value

Error message

Error decoding historicalPlanStatistics value

What it means

HistoricalStatisticsSerde is a Redis codec that deserializes Redis byte values into HistoricalPlanStatistics using the Drift Thrift codec with the BINARY protocol. When the underlying ThriftProtocolUtils.read call fails with a ThriftProtocolException, it is wrapped in a RedisProviderSerdeException with this message. This means the bytes read from Redis are not a valid Thrift-BINARY-encoded HistoricalPlanStatistics payload.

Source

Thrown at redis-hbo-provider/src/main/java/com/facebook/presto/statistic/HistoricalStatisticsSerde.java:78

        SliceOutput dynamicSliceOutput = new DynamicSliceOutput(ESTIMATED_BUFFER_SIZE_BYTES);
        try {
            ThriftProtocolUtils.write(historicalPlanStatistics, writeCodec, Protocol.BINARY, dynamicSliceOutput);
            return ByteBuffer.wrap(dynamicSliceOutput.slice().getBytes());
        }
        catch (ThriftProtocolException e) {
            throw new RedisProviderSerdeException("Error encoding historicalPlanStatistics value", e);
        }
    }

    @Override
    public HistoricalPlanStatistics decodeValue(ByteBuffer byteBuffer)
    {
        ThriftCodec<HistoricalPlanStatistics> readCodec = thriftCodecManager.getCodec(HistoricalPlanStatistics.class);
        try {
            return ThriftProtocolUtils.read(readCodec, Protocol.BINARY, Slices.wrappedBuffer(byteBuffer).getInput());
        }
        catch (ThriftProtocolException e) {
            throw new RedisProviderSerdeException("Error decoding historicalPlanStatistics value", e);
        }
    }
}

View on GitHub (pinned to 55bb57d202)

Solutions

1. Flush the Redis keyspace (or delete the offending plan-hash keys) so stale/incompatible encoded values are repopulated by encodeValue with the current schema.
2. Verify values were written with Protocol.BINARY via ThriftProtocolUtils.write and not by another serializer/protocol; re-encode with the current writer.
3. Check for a Presto/Drift version skew between the writer and reader clusters and align versions so the Thrift schema matches.
4. Inspect the wrapped ThriftProtocolException (getCause) for the exact decode offset/field to confirm corruption vs schema mismatch.
5. Add entry validation (try decode, skip/log on RedisProviderSerdeException) so a single bad entry does not break statistics collection.

Example fix

// before
return ThriftProtocolUtils.read(readCodec, Protocol.BINARY, Slices.wrappedBuffer(byteBuffer).getInput());
// after
if (byteBuffer == null || !byteBuffer.hasRemaining()) {
    return null; // skip empty/invalid Redis value instead of failing decode
}
return ThriftProtocolUtils.read(readCodec, Protocol.BINARY, Slices.wrappedBuffer(byteBuffer).getInput());

// validate before trusting Redis bytes
if (byteBuffer == null || !byteBuffer.hasRemaining()) {
    throw new IllegalArgumentException("Empty value for historicalPlanStatistics");
}
HistoricalPlanStatistics stats;
try {
    stats = ThriftProtocolUtils.read(readCodec, Protocol.BINARY, Slices.wrappedBuffer(byteBuffer).getInput());
}
catch (ThriftProtocolException e) {
    throw new RedisProviderSerdeException("Error decoding historicalPlanStatistics value", e);
}

private static boolean isDecodableValue(ByteBuffer buf)
{
    return buf != null && buf.hasRemaining();
}

try {
    HistoricalPlanStatistics stats = serde.decodeValue(byteBuffer);
    useStats(stats);
}
catch (RedisProviderSerdeException e) {
    log.warn(e, "Skipping undecodable historicalPlanStatistics entry (cause: %s)", e.getCause());
    redis.delete(key); // drop corrupt/stale entry so it is re-encoded
}

When it happens

Trigger: Redis returns bytes that cannot be parsed as a Thrift BINARY encoding of HistoricalPlanStatistics: truncated/corrupted records, values written with a different Thrift protocol (e.g. COMPACT) or an incompatible HistoricalPlanStatistics schema version, stale entries persisted by an older Presto build, or manual writes to the Redis keyspace by another tool.

Common situations: Upgrading Presto where the HistoricalPlanStatistics Thrift schema changed but old Redis entries remain; pointing a cluster at a Redis DB populated by a different component or protocol; Redis data corruption or partial writes; keys written by a test/other service using a non-Binary protocol.

Related errors

• NOT_SUPPORTED
• Type %s is not supported
• Role must be present for the selected role type:
• Expected a Class type for javaType, but got:
• Can not deserialize the data

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/d50d2584c22f3373. Report an issue: GitHub.