apache/beam · error · RuntimeException

Unexpected exception fetching database schema.

Error message

Unexpected exception fetching database schema.

What it means

getOneRecord reflectsively instantiates and drives a Debezium connector task. Reflective calls declare checked exceptions (NoSuchMethodException, InterruptedException, InvocationTargetException, IllegalAccessException, InstantiationException); any of these is wrapped in this RuntimeException. It signals a harness/reflection problem rather than a database failure.

Source

Thrown at sdks/java/io/debezium/src/main/java/org/apache/beam/io/debezium/KafkaSourceConsumerFn.java:180

      int loops = 0;
      while (records.size() == 0) {
        if (loops > 3) {
          throw new RuntimeException("could not fetch database schema");
        }
        records = task.poll();
        // Waiting for the Database snapshot to finish.
        Thread.sleep(2000);
        loops += 1;
      }
      task.stop();
      connector.stop();
      return records.get(0);
    } catch (NoSuchMethodException
        | InterruptedException
        | InvocationTargetException
        | IllegalAccessException
        | InstantiationException e) {
      throw new RuntimeException("Unexpected exception fetching database schema.", e);
    }
  }

  void register(RestrictionTracker<OffsetHolder, Map<String, Object>> tracker) {
    restrictionTrackers.put(this.getHashCode(), tracker);
  }

  void reset() {
    restrictionTrackers.remove(this.getHashCode());
  }

  @GetInitialWatermarkEstimatorState
  public Instant getInitialWatermarkEstimatorState(@Timestamp Instant currentElementTimestamp) {
    return currentElementTimestamp;
  }

  @NewWatermarkEstimator
  public WatermarkEstimator<Instant> newWatermarkEstimator(

View on GitHub (pinned to 12126d8942)

Solutions

Align the Debezium connector version with the one Apache Beam's DebeziumIO was built against.
Check the classpath for duplicate/conflicting Debezium artifacts (mvn dependency:tree).
Read the cause chain (the wrapped exception) to identify which reflective call failed.
Avoid interrupting the worker thread mid-schema-fetch; retry the pipeline.

Example fix

// before
debeziumVersion = 1.9.7.Final // with Beam built for 1.6.x
// after
match the io.debezium version to your Beam release's dependency

Defensive patterns

Strategy: try-catch

Validate before calling

Class.forName("io.debezium.connector.mysql.MySqlConnectorTask", true, loader);
// fails fast if Debezium classes are absent/conflicting

Try / catch

try {
  pipeline.run().waitUntilFinish();
} catch (RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().contains("Unexpected exception fetching database schema")) {
    LOG.error("Check Debezium version/classpath; cause:", e.getCause());
  }
}

Prevention

Match io.debezium version to your Beam release's dependency
Run mvn dependency:tree to detect conflicting Debezium artifacts
Keep the schema-fetch phase isolated from shutdown/interruption

When it happens

Trigger: The Connector/Task class lacks an expected method or no-arg constructor, the polling thread is interrupted, or reflection-based invocation fails while fetching one record.

Common situations: Debezium version incompatibility (class/API changed so reflection targets are missing), classpath conflicts with multiple Debezium versions, thread interruption during pipeline shutdown.

Understand the failure class

Background: "not installed", "pip install", "required for": how missing-dependency errors surface across open-source libraries — this error's family across 34 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/bdd824966492d039. Report an issue: GitHub.