What causes "Failed to get table schema for table: " in apache/beam?

Check the wrapped cause for the real error (connection refused, auth, unknown table)

apache/beam · error · RuntimeException

Failed to get table schema for table:

Error message

Failed to get table schema for table:

What it means

ClickHouseIO fetches the target table's schema (columns and types) at expansion time to map it to a Beam schema. Any exception during this metadata lookup — connection failure, unknown table, insufficient privileges, unsupported type — is caught and rethrown as RuntimeException("Failed to get table schema for table: " + table) with the original cause attached.

Source

Thrown at sdks/java/io/clickhouse/src/main/java/org/apache/beam/sdk/io/clickhouse/ClickHouseIO.java:784

            }
            DefaultType defaultType = DefaultType.parse(defaultTypeStr).orElse(null);

            Object defaultValue;
            if (DefaultType.DEFAULT.equals(defaultType)
                && !Strings.isNullOrEmpty(defaultExpression)) {
              defaultValue = ColumnType.parseDefaultExpression(columnType, defaultExpression);
            } else {
              defaultValue = null;
            }

            columns.add(TableSchema.Column.of(name, columnType, defaultType, defaultValue));
          }
        }
      }

      return TableSchema.of(columns.toArray(new TableSchema.Column[0]));
    } catch (Exception e) {
      throw new RuntimeException("Failed to get table schema for table: " + table, e);
    }
  }

  @VisibleForTesting
  static String buildClientName(Properties properties) {
    String beamAgent =
        String.format("Apache Beam/%s", ReleaseInfo.getReleaseInfo().getSdkVersion());
    String existingClientName = properties.getProperty("client_name");
    if (!Strings.isNullOrEmpty(existingClientName)) {
      return beamAgent + " " + existingClientName;
    }
    return beamAgent;
  }

  static String quoteIdentifier(String identifier) {
    String backslash = "\\\\";
    String quote = "\"";

View on GitHub (pinned to 12126d8942)

Solutions

Check the wrapped cause for the real error (connection refused, auth, unknown table)
Verify the table/database name and that it exists: run DESCRIBE TABLE in a ClickHouse client
Confirm the JDBC user has privileges to read the table metadata
Validate connectivity from the Beam environment (host/port/TLS settings)

Example fix

// before
ClickHouseIO.<Row>read("jdbc:clickhouse://host:8123/default", "custmers")
// after
ClickHouseIO.<Row>read("jdbc:clickhouse://host:8123/default", "customers") // table exists and user has grants

Defensive patterns

Strategy: try-catch

Validate before calling

// verify the table exists and is describable before expansion
try (Statement s = DriverManager.getConnection(jdbcUrl).createStatement()) {
  s.executeQuery("DESCRIBE TABLE " + table); // throws early if missing/no grants
}

Try / catch

try {
  ClickHouseIO.<Row>read(jdbcUrl, table);
} catch (RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("Failed to get table schema")) {
    LOG.error("Schema fetch failed for {} — cause: {}", table, e.getCause());
  }
  throw e;
}

Prevention

Validate table/database names against the target environment before deploying
Grant the JDBC user metadata/SELECT privileges on the table
Test connectivity from the Beam worker network (host, port, TLS)
Re-check the schema after ClickHouse version upgrades for unsupported types

When it happens

Trigger: Calling ClickHouseIO.read/write expansion against a table that does not exist, a database the user cannot read, an unreachable ClickHouse endpoint, or a table with column types the mapper cannot handle.

Common situations: Typo in table or database name; wrong environment URL (staging vs prod); user lacking SELECT/DESCRIBE grants; table created after the pipeline was built, or against a replica that is down.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f222e90973467da5. Report an issue: GitHub.