apache/beam · error · IllegalStateException

Failed to validate %s

Error message

Failed to validate %s

What it means

TFRecordReadSchemaTransformConfiguration's validate step matches the configured file pattern via FileSystems.match(); an IOException there is wrapped in an IllegalStateException prefixed with an invalid-config message. It means the configured input could not be validated as accessible files before pipeline execution.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/io/TFRecordReadSchemaTransformConfiguration.java:57

 * <p><b>Internal only:</b> This class is actively being worked on, and it will likely change. We
 * provide no backwards compatibility guarantees, and it should not be implemented outside the Beam
 * repository.
 */
@DefaultSchema(AutoValueSchema.class)
@AutoValue
public abstract class TFRecordReadSchemaTransformConfiguration implements Serializable {

  public void validate() {
    String invalidConfigMessage = "Invalid TFRecord Read configuration: ";

    if (getValidate()) {
      String filePattern = getFilePattern();
      try {
        MatchResult matches = FileSystems.match(filePattern);
        checkState(
            !matches.metadata().isEmpty(), "Unable to find any files matching %s", filePattern);
      } catch (IOException e) {
        throw new IllegalStateException(
            String.format(invalidConfigMessage + "Failed to validate %s", filePattern), e);
      }
    }

    ErrorHandling errorHandling = getErrorHandling();
    if (errorHandling != null) {
      checkArgument(
          !Strings.isNullOrEmpty(errorHandling.getOutput()),
          "%sOutput must not be empty if error handling specified.",
          invalidConfigMessage);
    }
  }

  /** Instantiates a {@link TFRecordReadSchemaTransformConfiguration.Builder} instance. */
  public static TFRecordReadSchemaTransformConfiguration.Builder builder() {
    return new AutoValue_TFRecordReadSchemaTransformConfiguration.Builder();
  }

View on GitHub (pinned to 12126d8942)

Solutions

1. Correct the filePattern value in the transform configuration and inspect the wrapped IOException cause.
2. Ensure the filesystem implementation for the scheme is on the classpath (e.g. beam-sdks-java-io-google-cloud-platform).
3. Route failures to the errorHandling output or disable validation if the files appear only at runtime.

Example fix

// before
.filePattern("gs://bucket/no-such-dir/*.tfrecord") // plus connectivity failure

// after
.filePattern("gs://bucket/data/*.tfrecord") // reachable, valid path

MatchResult mr = FileSystems.match(cfg.getFilePattern());
if (mr.metadata().isEmpty()) {
  throw new IllegalArgumentException("TFRecord SchemaTransform filePattern matched no files: " + cfg.getFilePattern());
}

try { transform.validate(); } catch (IllegalStateException e) { if (e.getCause() instanceof IOException) { log.error("FilePattern validation failed: {}", cfg.getFilePattern(), e.getCause()); } throw e; }

When it happens

Trigger: Constructing a TFRecord read SchemaTransform with a filePattern that FileSystems.match cannot resolve — unregistered scheme, unreachable filesystem, or I/O/auth error; requires error handling/validate path in the transform's validate().

Common situations: YAML/SQL pipeline config pointing at a typo'd gs:// path; missing GCS connector dependency; permissions or credentials failing at graph-construction validation time.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• Failed to validate %s
• Pipeline update will not be possible because the following t
• One or more ErrorHandlers aren't closed, and this pipeline c
• Failed to validate transform %s
• The relative error must be positive

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/30167e8de82013df. Report an issue: GitHub.