apache/beam · error · IllegalArgumentException

Metadata database can not be null

Error message

Metadata database can not be null

What it means

DaoFactory's constructor also requires metadataSpannerConfig.getDatabaseId() to be non-null. The metadata database (which stores change stream partitions and watermarks) must be specified; otherwise IllegalArgumentException is thrown right after the instance check.

Solutions

1. Set the metadata database on the configuration, e.g. withMetadataDatabase("change-stream-metadata").
2. If metadata should live in the same database as the change stream source, explicitly set that database id in the metadata config.
3. Validate the assembled SpannerConfig (instanceId and databaseId non-null) before constructing DaoFactory.

Example fix

// before
.withMetadataInstance("my-instance")
// after
.withMetadataInstance("my-instance")
.withMetadataDatabase("my-metadata-db")

if (metadataSpannerConfig.getDatabaseId() == null) {
  throw new IllegalArgumentException("Set withMetadataDatabase() before expansion");
}

When it happens

Trigger: Building DaoFactory with a metadata SpannerConfig that has an instanceId but no databaseId — the metadata database option was omitted in the transform configuration.

Common situations: Configuration where withMetadataInstance was called but withMetadataDatabase was forgotten; environment-driven config assembly where the metadata database id variable is empty or unset.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• Metadata instance can not be null
• Null collection element type at field
• Average partition bytes size has not been initialized…
• Both query and table cannot be specified at the same time…
• cannot encode a null BitSet

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/179236792761b25c. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/dao/DaoFactory.java:86

   * @param rpcPriority the priority of the requests made by the DAO queries
   * @param jobName the name of the running job
   */
  public DaoFactory(
      SpannerConfig changeStreamSpannerConfig,
      String changeStreamName,
      List<String> tvfNameList,
      SpannerConfig metadataSpannerConfig,
      PartitionMetadataTableNames partitionMetadataTableNames,
      RpcPriority rpcPriority,
      String jobName,
      Dialect spannerChangeStreamDatabaseDialect,
      Dialect metadataDatabaseDialect,
      boolean isMutableChangeStream) {
    if (metadataSpannerConfig.getInstanceId() == null) {
      throw new IllegalArgumentException("Metadata instance can not be null");
    }
    if (metadataSpannerConfig.getDatabaseId() == null) {
      throw new IllegalArgumentException("Metadata database can not be null");
    }
    this.changeStreamSpannerConfig = changeStreamSpannerConfig;
    this.changeStreamName = changeStreamName;
    this.tvfNameList =
        tvfNameList == null ? ChangeStreamsConstants.DEFAULT_TVF_NAME_LIST : tvfNameList;
    this.metadataSpannerConfig = metadataSpannerConfig;
    this.partitionMetadataTableNames = partitionMetadataTableNames;
    this.rpcPriority = rpcPriority;
    this.jobName = jobName;
    this.spannerChangeStreamDatabaseDialect = spannerChangeStreamDatabaseDialect;
    this.metadataDatabaseDialect = metadataDatabaseDialect;
    this.isMutableChangeStream = isMutableChangeStream;
  }

  /** Returns the tvf name list. */
  public List<String> getTvfNameList() {
    return this.tvfNameList;
  }

View on GitHub (pinned to 12126d8942)