apache/beam · error

Failed to fetch partition_mode for change stream

Error message

Failed to fetch partition_mode for change stream '{}', dialect={} - will propagate exception

What it means

SpannerIO failed to fetch the partition_mode configuration for a change stream (checking whether the change stream supports partitioned reads via the given database dialect). The failure is logged with a stack trace and the RuntimeException is rethrown so the caller still sees the error and the read fails rather than silently defaulting.

Solutions

1. Verify the change stream name exists in the database (SELECT * FROM INFORMATION_SCHEMA.CHANGE_STREAMS)
2. Check the database dialect (GoogleSQL vs PostgreSQL) — partition_mode metadata may not exist for your dialect
3. Inspect the rethrown exception's cause for the underlying Spanner status and fix permissions/connectivity
4. Retry if the failure was transient Spanner unavailability

Example fix

// before
String changeStreamName = "myStream";
// after
// confirm existence first:
// SELECT change_stream_name FROM INFORMATION_SCHEMA.CHANGE_STREAMS WHERE change_stream_name = 'myStream';
String changeStreamName = "myStream"; // must exist for the configured dialect

// before building the SpannerIO read
boolean exists = querySpanner(
  "SELECT change_stream_name FROM INFORMATION_SCHEMA.CHANGE_STREAMS WHERE change_stream_name = ?",
  changeStreamName);
if (!exists) throw new IllegalArgumentException("unknown change stream: " + changeStreamName);

try {
  String mode = fetchPartitionMode(changeStreamName, dialect);
} catch (RuntimeException e) {
  throw new IllegalStateException(
    "Cannot fetch partition_mode for change stream " + changeStreamName + " (dialect " + dialect + ")", e);
}

When it happens

Trigger: A RuntimeException is thrown while querying Spanner metadata (e.g., INFORMATION_SCHEMA or change stream options) for the named change stream: change stream does not exist, database unavailable, permission denied, or dialect-specific query not supported (e.g., querying partition_mode on a dialect that lacks it).

Common situations: Misspelled change stream name, using PostgreSQL-dialect Spanner databases where partition_mode metadata is unavailable, IAM lacking spanner.database read permissions, transient Spanner unavailability during job setup.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• Error while parsing the DataChangeRecord
• failed to initialise Spanner client
• MetadataDatabase can't be empty
• MetadataInstance can't be empty
• SpannerException (rethrown from operation failure/timeout…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/6ce15d49a8c9e9f0. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.java:3135

                    "select option_value\n"
                        + "from information_schema.change_stream_options\n"
                        + "where change_stream_name = @changeStreamName and  option_name = 'partition_mode'")
                .bind("changeStreamName")
                .to(changeStreamName)
                .build();
      }
      ResultSet resultSet = tx.executeQuery(statement);
      while (resultSet.next()) {
        String value = resultSet.getString(0);
        if (value != null) {
          return value;
        }
      }
      return "";
    } catch (RuntimeException e) {
      // Log the failure (with stack trace) but rethrow so the caller still observes
      // the error.
      LOG.warn(
          "Failed to fetch partition_mode for change stream '{}', dialect={} - will propagate exception",
          changeStreamName,
          dialect,
          e);
      throw e;
    }
  }
}

View on GitHub (pinned to 12126d8942)