prestodb/presto · error · PrestoException

GENERIC_INTERNAL_ERROR

GENERIC_INTERNAL_ERROR

Error message

No avro record found

What it means

AvroRowDecoder.decodeRow wraps the raw Avro bytes in a DataFileStream and expects exactly one GenericRecord. This GENERIC_INTERNAL_ERROR is thrown when the stream is a valid Avro container but contains zero records, meaning the payload has only a schema header and no data. The decoder treats an empty record stream as a protocol violation it cannot recover from.

Source

Thrown at presto-record-decoder/src/main/java/com/facebook/presto/decoder/avro/AvroRowDecoder.java:65

                .collect(toImmutableMap(identity(), this::createColumnDecoder));
    }

    private AvroColumnDecoder createColumnDecoder(DecoderColumnHandle columnHandle)
    {
        return new AvroColumnDecoder(columnHandle);
    }

    @Override
    public Optional<Map<DecoderColumnHandle, FieldValueProvider>> decodeRow(byte[] data, Map<String, String> dataMap)
    {
        GenericRecord avroRecord;
        DataFileStream<GenericRecord> dataFileReader = null;
        try {
            // Assumes producer uses DataFileWriter or data comes in this particular format.
            // TODO: Support other forms for producers
            dataFileReader = new DataFileStream<>(new ByteArrayInputStream(data), avroRecordReader);
            if (!dataFileReader.hasNext()) {
                throw new PrestoException(GENERIC_INTERNAL_ERROR, "No avro record found");
            }
            avroRecord = dataFileReader.next();
            if (dataFileReader.hasNext()) {
                throw new PrestoException(GENERIC_INTERNAL_ERROR, "Unexpected extra record found");
            }
        }
        catch (Exception e) {
            throw new PrestoException(GENERIC_INTERNAL_ERROR, "Decoding Avro record failed.", e);
        }
        finally {
            closeQuietly(dataFileReader);
        }

        return Optional.of(columnDecoders.entrySet().stream()
                .collect(toImmutableMap(
                        Map.Entry::getKey,
                        entry -> entry.getValue().decodeField(avroRecord))));
    }

View on GitHub (pinned to 55bb57d202)

Solutions

1. Inspect the raw payload bytes and confirm the producer is actually writing a record, not an empty DataFileWriter output
2. Verify the producer-side Avro serialization path (DataFileWriter append + flush/close) actually writes the record before committing
3. Check for message truncation or corruption between producer and consumer (broker retention, max message size, proxy buffering)
4. If empty payloads are legitimate in your pipeline, filter them before calling decodeRow instead of routing them to the decoder

Example fix

// before (producer): DataFileWriter created but record appended conditionally, file closed with 0 records
// after (producer):
if (record != null) {
    writer.append(record);
}
writer.close();
// consumer side: skip empty payloads before decode
if (data == null || data.length == 0) { return Optional.empty(); }

boolean isNonEmptyAvroContainer(byte[] data) {
    return data != null && data.length > 4
        && data[0]=='O' && data[1]=='b' && data[2]=='j' && data[3]==1;
}

boolean hasAtLeastOneRecord(byte[] data) throws IOException {
    try (DataFileStream<GenericRecord> r =
             new DataFileStream<>(new ByteArrayInputStream(data), new GenericDatumReader<>())) {
        return r.hasNext();
    }
}

try { decoder.decodeRow(...) }
catch (PrestoException e) {
    if (e.getErrorCode().getName().equals("GENERIC_INTERNAL_ERROR")) {
        log.warn("Skipping empty/invalid avro payload"); return Optional.empty();
    }
    throw e;
}

When it happens

Trigger: Kafka message payload contains a valid Avro DataFile header but no records (empty container); producer wrote a file with zero rows; payload truncated to just the header portion.

Common situations: Producer misconfiguration emitting empty Avro files; a heartbeat/no-op message serialized as an empty container; truncation or corruption dropping the record block while keeping the header; producer switching formats mid-topic so old consumers see empty streams.

Related errors

• KAFKA_SCHEMA_ERROR
• Failed to append record
• Failed to close ByteArrayOutputStream
• BIGQUERY_UNSUPPORTED_TYPE_FOR_LONG
• BIGQUERY_UNSUPPORTED_TYPE_FOR_VARBINARY

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/a4ea513b6da2be51. Report an issue: GitHub.