apache/beam · error · IllegalArgumentException

BigQueryIO.Write transforms cannot be converted to a…

Error message

BigQueryIO.Write transforms cannot be converted to a portable row based config due to 'withSchemaFromView' property being set. Please retry without setting this property when configuring your transform

What it means

BigQueryIOTranslation.toConfigRow converts a BigQueryIO.Write transform into a portable Row-based configuration for cross-language pipelines. The withSchemaFromView property stores a reference to a PCollectionView bound to the current pipeline instance, which cannot be serialized portably, so the translation throws IllegalArgumentException and asks the caller to configure the transform without it.

Solutions

Remove .withSchemaFromView(...) and instead set a static schema with .withJsonSchema(...) or .withSchema(...) / fromValue(...).
If schemas must be dynamic per destination, use DynamicDestinations with getSchema() instead of a schema view.
Keep the pipeline entirely on a non-portable Java runner if schema-from-view is essential (it is only usable in non-portable pipelines).
Upgrade Beam to a version whose portable path supports your configuration pattern, if available.

Example fix

// before
BigQueryIO.write().withSchemaFromView(schemaView)
// after
BigQueryIO.write().to(new DynamicDestinations<TableRow, String>() {
  public TableDestination getDestination(TableRow row) { ... }
  public TableSchema getSchema(String dest) { ... }
})

Defensive patterns

Strategy: validation

Validate before calling

if (writeTransform.getSchemaFromView() != null && isPortableOrCrossLanguagePipeline()) {
  throw new IllegalArgumentException("withSchemaFromView is not portable; use DynamicDestinations or a static schema");
}

Type guard

static boolean isPortableSafe(Write<?> w) {
  return w.getSchemaFromView() == null;
}

Try / catch

try {
  row = BigQueryIOTranslation.toConfigRow(transform, ...);
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("withSchemaFromView")) {
    throw new IllegalStateException("Reconfigure the transform with a static schema or DynamicDestinations for portability", e);
  } else { throw e; }
}

Prevention

Avoid withSchemaFromView entirely in cross-language or portable pipelines.
Use DynamicDestinations.getSchema() for runtime-computed schemas.
Document schema strategy in shared transform factories so downstream languages stay compatible.

When it happens

Trigger: Expanding a BigQueryIO.Write built with .withSchemaFromView(pCollectionView) inside a portable/cross-language (e.g. Python runner via expansion service) pipeline, when toConfigRow is invoked during translation.

Common situations: Java pipelines consumed from Python via cross-language transforms; dynamic per-window schemas configured with SchemaFromView then submitted to a portable runner; template/URN-based translation paths that must round-trip the transform config.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/176777d8272829a5. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIOTranslation.java:542

      }
      Object formatRecordOnFailureFunction = transform.getFormatRecordOnFailureFunction();
      if (formatRecordOnFailureFunction != null) {
        fieldValues.put(
            "format_record_on_failure_function", toByteArray(formatRecordOnFailureFunction));
      }
      Object avroRowWriterFactory = transform.getAvroRowWriterFactory();
      if (avroRowWriterFactory != null) {
        fieldValues.put("avro_row_writer_factory", toByteArray(avroRowWriterFactory));
      }
      fieldValues.put("use_avro_logical_types", transform.getUseAvroLogicalTypes());
      Object dynamicDestinations = transform.getDynamicDestinations();
      if (dynamicDestinations != null) {
        fieldValues.put("dynamic_destinations", toByteArray(dynamicDestinations));
      }
      if (transform.getSchemaFromView() != null) {
        // Property 'getSchemaFromView' cannot be used in a portable way across pipelines since it
        // is bound to PCollections generated for the current pipeline instance.
        throw new IllegalArgumentException(
            "BigQueryIO.Write transforms cannot be converted to a "
                + "portable row based config due to 'withSchemaFromView' property being set. Please "
                + "retry without setting this property when configuring your transform");
      }
      ValueProvider<String> jsonSchema = transform.getJsonSchema();
      if (jsonSchema != null) {
        fieldValues.put("json_schema", jsonSchema.get());
      }
      ValueProvider<String> jsonTimePartitioning = transform.getJsonTimePartitioning();
      if (jsonTimePartitioning != null) {
        fieldValues.put("json_time_partitioning", toByteArray(jsonTimePartitioning.get()));
      }
      ValueProvider<String> jsonClustering = transform.getJsonClustering();
      if (jsonClustering != null) {
        fieldValues.put("clustering", jsonClustering.get());
      }
      if (transform.getCreateDisposition() != null) {
        fieldValues.put("create_disposition", toByteArray(transform.getCreateDisposition()));

View on GitHub (pinned to 12126d8942)