apache/beam · error · IllegalStateException

Need to set the filepattern of a TFRecordIO.Read transform

Error message

Need to set the filepattern of a TFRecordIO.Read transform

What it means

TFRecordIO.Read.expand() throws this IllegalStateException when the transform's filepattern was never set. TFRecordIO needs a file glob or path to read from; without it there is nothing to configure the source with, so validation fails fast before pipeline submission.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/io/TFRecordIO.java:183

      return withCompression(compressionType.canonical);
    }

    /**
     * Returns a transform for reading TFRecord files that decompresses all input files using the
     * specified compression type.
     *
     * <p>If no compression type is specified, the default is {@link Compression#AUTO}. In this
     * mode, the compression type of the file is determined by its extension via {@link
     * Compression#detect(String)}.
     */
    public Read withCompression(Compression compression) {
      return toBuilder().setCompression(compression).build();
    }

    @Override
    public PCollection<byte[]> expand(PBegin input) {
      if (getFilepattern() == null) {
        throw new IllegalStateException(
            "Need to set the filepattern of a TFRecordIO.Read transform");
      }

      if (getValidate()) {
        checkState(getFilepattern().isAccessible(), "Cannot validate with a RVP.");
        try {
          MatchResult matches = FileSystems.match(getFilepattern().get());
          checkState(
              !matches.metadata().isEmpty(),
              "Unable to find any files matching %s",
              getFilepattern().get());
        } catch (IOException e) {
          throw new IllegalStateException(
              String.format("Failed to validate %s", getFilepattern().get()), e);
        }
      }

      return input.apply("Read", org.apache.beam.sdk.io.Read.from(getSource()));

View on GitHub (pinned to 12126d8942)

Solutions

1. Call .from("path/or/glob/*.tfrecords") on the TFRecordIO.read() builder before applying the transform.
2. Check that the variable holding the filepattern is non-null at transform construction time.

Example fix

// before
pipeline.apply(TFRecordIO.read());

// after
pipeline.apply(TFRecordIO.read().from("gs://my-bucket/data/*.tfrecord"));

if (filePattern == null || filePattern.isEmpty()) {
  throw new IllegalArgumentException("TFRecordIO filepattern must be set via .from(...)");
}
TFRecordIO.read().from(filePattern);

try { pipeline.apply(TFRecordIO.read().from(filePattern)); } catch (IllegalStateException e) { if (e.getMessage().contains("filepattern")) { log.error("Missing TFRecordIO filepattern"); } throw e; }

When it happens

Trigger: Applying TFRecordIO.read() without calling .from("gs://bucket/path/*.tfrecord") on the builder.

Common situations: Building the transform programmatically where the from() call was skipped; passing a null variable holding the path; conditional code paths that never set the filepattern.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• expected %d, but got %d
• Failed to validate %s
• Mismatch of length mask when reading a record. Expected %d b
• length overflow %d
• Mismatch of data mask when reading a record. Expected %d but

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/6ea2198a62ae7d65. Report an issue: GitHub.