apache/beam · error · SpannerSchemaRetrievalException

Exception while trying to retrieve schema

Error message

Exception while trying to retrieve schema

What it means

The outer catch-all of SpannerTableSourceDef.getBeamSchema() wraps any Exception thrown while reading the table or building the Beam schema into SpannerSchemaRetrievalException("Exception while trying to retrieve schema", e). This includes Spanner API errors (NOT_FOUND, PERMISSION_DENIED, DEADLINE_EXCEEDED), session errors, and mapping failures — the real cause is on the exception's cause chain.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerTableSourceDef.java:59

    this.config = config;
    this.columns = columns;
  }

  /** {@inheritDoc} */
  @Override
  public Schema getBeamSchema() {
    Schema beamSchema;
    try (SpannerAccessor spannerAccessor = SpannerAccessor.getOrCreate(config)) {
      try (ReadContext readContext = spannerAccessor.getDatabaseClient().singleUse()) {
        ResultSet result = readContext.read(table, KeySet.all(), columns, Options.limit(1));
        if (result.next()) {
          beamSchema = structTypeToBeamRowSchema(result.getMetadata().getRowType(), true);
        } else {
          throw new SpannerSchemaRetrievalException("Cannot find Spanner table.");
        }
      }
    } catch (Exception e) {
      throw new SpannerSchemaRetrievalException("Exception while trying to retrieve schema", e);
    }
    return beamSchema;
  }
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Log/inspect the cause of SpannerSchemaRetrievalException for the concrete SpannerException code.
2. Fix SpannerConfig (projectId/instanceId/databaseId/table name) to match an existing table.
3. Grant the caller spanner.databases.read (and related) IAM roles.
4. Retry on transient codes (DEADLINE_EXCEEDED, UNAVAILABLE); check Google Cloud status if persistent.

Example fix

// before
PCollection<Row> rows = p.apply(SpannerRead.of(wrongConfig).withTable("users"));
// after
SpannerConfig cfg = SpannerConfig.create().withProjectId("my-project").withInstanceId("prod").withDatabaseId("app");
PCollection<Row> rows = p.apply(SpannerRead.of(cfg).withTable("users"));

// Pre-flight existence + permission probe
try (ResultSet rs = dbClient.singleUse().executeQuery(Statement.of("SELECT 1 FROM " + table + " LIMIT 1"))) { rs.next(); }

try { rows = p.apply(SpannerRead.of(config).withTable(table)); } catch (SpannerSchemaRetrievalException e) {
  Throwable cause = e.getCause();
  if (cause instanceof SpannerException) { switch (((SpannerException) cause).getErrorCode()) { case NOT_FOUND: /* fix config */ break; case PERMISSION_DENIED: /* fix IAM */ break; case UNAVAILABLE: case DEADLINE_EXCEEDED: /* retry */ break; default: throw e; } } else throw e;
}

When it happens

Trigger: Any failure during getBeamSchema() other than the explicit empty-result case: nonexistent table (SpannerException NOT_FOUND), insufficient IAM permissions, transient Spanner unavailability, timeouts, or structTypeToBeamRowSchema mapping errors.

Common situations: Wrong instance/database/table name in SpannerConfig; service account without spanner.databases.read; Spanner incident or network partition; unsupported column types failing schema mapping.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

• Null collection element type at field {}
• Exception while trying to retrieve schema
• Invalid ARRAY type: + originalSpannerType
• Unknown spanner type + spannerType
• Cannot find Spanner table.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/6e694d060214be41. Report an issue: GitHub.