apache/beam · error · IllegalArgumentException

Metadata instance can not be null

Error message

Metadata instance can not be null

What it means

DaoFactory's constructor validates the metadata Spanner configuration before building DAOs. If metadataSpannerConfig has no instanceId, it immediately throws IllegalArgumentException because change stream metadata cannot be stored without a Spanner instance.

Solutions

1. Set the metadata instance on the config, e.g. withMetadataInstance("my-instance") on the SpannerChangeStreamReadSchemaTransformConfiguration.
2. If metadata should live in the same instance as the source database, explicitly copy that instance id into the metadata config.
3. Check how the SpannerConfig is constructed programmatically and ensure instanceId is non-null before calling new DaoFactory(...).

Example fix

// before
SpannerConfig metaConfig = SpannerConfig.builder().setProjectId("p").setDatabaseId("meta-db").build();
// after
SpannerConfig metaConfig = SpannerConfig.builder().setProjectId("p").setInstanceId("my-instance").setDatabaseId("meta-db").build();

if (metadataSpannerConfig.getInstanceId() == null) {
  throw new IllegalArgumentException("Set withMetadataInstance() before expansion");
}

When it happens

Trigger: Constructing DaoFactory (e.g. from SpannerChangestreamsReadSchemaTransformProvider expansion) with a metadata configuration whose instanceId was never set — typically a SpannerConfig built with only projectId/databaseId.

Common situations: Users configuring the change stream read with only the source database settings and forgetting the separate metadata instance/database options; programmatic config building where setMetadataInstance() was skipped.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• Metadata database can not be null
• Null collection element type at field
• Average partition bytes size has not been initialized…
• Both query and table cannot be specified at the same time…
• cannot encode a null BitSet

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/01b9f4486fbce479. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/dao/DaoFactory.java:83

   * @param metadataSpannerConfig the metadata tables configuration
   * @param partitionMetadataTableNames the names of the partition metadata ddl objects
   * @param tvfNameList the list of TVF names specified to query and union
   * @param rpcPriority the priority of the requests made by the DAO queries
   * @param jobName the name of the running job
   */
  public DaoFactory(
      SpannerConfig changeStreamSpannerConfig,
      String changeStreamName,
      List<String> tvfNameList,
      SpannerConfig metadataSpannerConfig,
      PartitionMetadataTableNames partitionMetadataTableNames,
      RpcPriority rpcPriority,
      String jobName,
      Dialect spannerChangeStreamDatabaseDialect,
      Dialect metadataDatabaseDialect,
      boolean isMutableChangeStream) {
    if (metadataSpannerConfig.getInstanceId() == null) {
      throw new IllegalArgumentException("Metadata instance can not be null");
    }
    if (metadataSpannerConfig.getDatabaseId() == null) {
      throw new IllegalArgumentException("Metadata database can not be null");
    }
    this.changeStreamSpannerConfig = changeStreamSpannerConfig;
    this.changeStreamName = changeStreamName;
    this.tvfNameList =
        tvfNameList == null ? ChangeStreamsConstants.DEFAULT_TVF_NAME_LIST : tvfNameList;
    this.metadataSpannerConfig = metadataSpannerConfig;
    this.partitionMetadataTableNames = partitionMetadataTableNames;
    this.rpcPriority = rpcPriority;
    this.jobName = jobName;
    this.spannerChangeStreamDatabaseDialect = spannerChangeStreamDatabaseDialect;
    this.metadataDatabaseDialect = metadataDatabaseDialect;
    this.isMutableChangeStream = isMutableChangeStream;
  }

  /** Returns the tvf name list. */

View on GitHub (pinned to 12126d8942)