apache/beam · error · IllegalStateException

Read session does not have Avro/Arrow schema set.

Error message

Read session does not have Avro/Arrow schema set.

What it means

BigQueryStorageReaderFactory.getReader chooses an Avro or Arrow reader based on which schema the Storage Read API ReadSession carries. If the session has neither an Avro nor Arrow schema set, no reader implementation is applicable, so an IllegalStateException is thrown. This indicates the ReadSession was created without the expected serialization options.

Solutions

1. Set an explicit data format when creating the source/session, e.g. .withMethod(DIRECT_READ).withFormat(DataFormat.AVRO) (or ARROW)
2. Log/inspect the ReadSession (readSession.hasAvroSchema()/hasArrowSchema()) immediately after creation to catch empty sessions early
3. Retry session creation — a transient backend issue can omit schema; recreate the ReadSession
4. Upgrade the google-cloud-bigquery-storage client and Beam GCP SDK to a version matching your DataFormat support

Example fix

// before
ReadSession session = createSession(request); // DataFormat not set
BigQueryStorageReader reader = BigQueryStorageReaderFactory.getReader(session);
// after
ReadSession session = createSession(request.toBuilder()
    .setReadOptions(ReadSession.TableReadOptions.newBuilder()
        .setArrowSerializationOptions(...)) // or set data format AVRO/ARROW
    .build());
if (!session.hasAvroSchema() && !session.hasArrowSchema()) {
  session = recreateSessionWithFormat(DataFormat.AVRO);
}
BigQueryStorageReader reader = BigQueryStorageReaderFactory.getReader(session);

if (readSession == null || (!readSession.hasAvroSchema() && !readSession.hasArrowSchema())) {
  throw new IllegalArgumentException("ReadSession must be created with AVRO or ARROW format");
}

boolean hasSerializableSchema(ReadSession s) {
  return s != null && (s.hasAvroSchema() || s.hasArrowSchema());
}

try {
  return BigQueryStorageReaderFactory.getReader(session);
} catch (IllegalStateException e) {
  // recreate the session with explicit DataFormat before failing
  session = recreateSessionWithFormat(DataFormat.AVRO);
  return BigQueryStorageReaderFactory.getReader(session);
}

When it happens

Trigger: Calling BigQueryStorageReaderFactory.getReader(readSession) with a ReadSession whose readOptions/serialization options did not set an avro_schema or arrow_schema — typically a session created with DataFormat unset, or a server response that omitted both schemas.

Common situations: Building a custom Storage Read API source and forgetting to set the format (AVRO/ARROW) in TableReadOptions; a BigQuery backend returning a session without the requested schema after a silent fallback; mixing Beam versions where DataFormat defaults changed.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• BigQuery storage source must be split before reading
• createRowWriter called when dynamicDestinations is null…
• createRowWriter called when schemaFactory is null; forgot…
• Table not found
• Unsupported timestamp precision for Storage Read API:

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4cf6a96739dd8874. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryStorageReaderFactory.java:33

 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.apache.beam.sdk.io.gcp.bigquery;

import com.google.cloud.bigquery.storage.v1.ReadSession;
import java.io.IOException;

class BigQueryStorageReaderFactory {

  private BigQueryStorageReaderFactory() {}

  public static BigQueryStorageReader getReader(ReadSession readSession) throws IOException {
    if (readSession.hasAvroSchema()) {
      return new BigQueryStorageAvroReader(readSession);
    } else if (readSession.hasArrowSchema()) {
      return new BigQueryStorageArrowReader(readSession);
    }
    throw new IllegalStateException("Read session does not have Avro/Arrow schema set.");
  }
}

View on GitHub (pinned to 12126d8942)