prestodb/presto · error · BigQueryException

BIGQUERY_ERROR_END_OF_AVRO_BUFFER

BIGQUERY_ERROR_END_OF_AVRO_BUFFER

Error message

Error determining the end of Avro buffer

What it means

The Avro record iterator's hasNext calls decode.isEnd() to check whether more records remain in the Avro buffer returned by BigQuery Storage; an IOException there is wrapped in BIGQUERY_ERROR_END_OF_AVRO_BUFFER. It means the underlying Avro stream/decoder failed while checking for more data, typically a transport/read failure mid-stream.

Source

Thrown at presto-bigquery/src/main/java/com/facebook/presto/plugin/bigquery/BigQueryResultPageSource.java:321

            implements Iterator<GenericRecord>
    {
        GenericDatumReader<GenericRecord> reader;
        BinaryDecoder decode;

        AvroBinaryIterator(Schema avroSchema, byte[] buffer)
        {
            this.reader = new GenericDatumReader<>(avroSchema);
            this.decode = new DecoderFactory().binaryDecoder(buffer, null);
        }

        @Override
        public boolean hasNext()
        {
            try {
                return !decode.isEnd();
            }
            catch (IOException e) {
                throw new BigQueryException(BIGQUERY_ERROR_END_OF_AVRO_BUFFER, "Error determining the end of Avro buffer", e);
            }
        }

        @Override
        public GenericRecord next()
        {
            try {
                return reader.read(null, decode);
            }
            catch (IOException e) {
                throw new BigQueryException(BIGQUERY_ERROR_READING_NEXT_AVRO_RECORD, "Error reading next Avro Record", e);
            }
        }
    }

    static class AvroDecimalConverter
    {
        private static final Conversions.DecimalConversion AVRO_DECIMAL_CONVERSION = new Conversions.DecimalConversion();

View on GitHub (pinned to 55bb57d202)

Solutions

1. Retry the query/split — the failure is often transient
2. Re-create the BigQuery read session and re-run the scan
3. Check network connectivity and any proxy/firewall between Presto and Google APIs
4. Check Google Cloud status for BigQuery Storage API incidents

// preflight: confirm the read session/stream is reachable before scanning
ReadSession session = bigQueryClient.createReadSession(readSessionRequest);
if (session.getStreamsCount() == 0) {
    throw new IllegalStateException("No streams returned for read session");
}

int attempts = 0;
while (attempts < 3) {
    try {
        return pageSource.getNextPage(...);
    } catch (BigQueryException e) {
        if (BIGQUERY_ERROR_END_OF_AVRO_BUFFER.getCode() == e.getErrorCode().getCode() && ++attempts < 3) {
            backoff(attempts); continue; // transient stream error
        }
        throw e;
    }
}

When it happens

Trigger: Reading rows from a BigQuery Storage read session when the underlying gRPC-to-Avro stream is interrupted or the decoder hits a corrupt/truncated buffer during isEnd().

Common situations: Network interruptions to Google APIs; read session expiring mid-scan; truncated responses; transient Google backend errors.

Related errors

• BIGQUERY_ERROR_READING_NEXT_AVRO_RECORD
• Failed to create Credentials from file
• BIGQUERY_UNSUPPORTED_TYPE_FOR_LONG
• BIGQUERY_UNSUPPORTED_TYPE_FOR_VARBINARY
• KAFKA_SCHEMA_ERROR

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/2192f6f831995b47. Report an issue: GitHub.