apache/beam · error · IOException

Error matching file spec %s: status %s

Error message

Error matching file spec %s: status %s

What it means

FileSystems.matchSingleFileSpec throws IOException when the underlying MatchResult status is neither OK nor NOT_FOUND, meaning the filesystem provider could not determine match results (e.g. an error occurred listing/matching).

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileSystems.java:198

  }

  /**
   * Returns the {@link Metadata} for a single file resource. Expects a resource specification
   * {@code spec} that matches a single result.
   *
   * @param spec a resource specification that matches exactly one result.
   * @return the {@link Metadata} for the specified resource.
   * @throws FileNotFoundException if the file resource is not found.
   * @throws IOException in the event of an error in the inner call to {@link #match}, or if the
   *     given spec does not match exactly 1 result.
   */
  public static Metadata matchSingleFileSpec(String spec) throws IOException {
    List<MatchResult> matches = FileSystems.match(Collections.singletonList(spec));
    MatchResult matchResult = Iterables.getOnlyElement(matches);
    if (matchResult.status() == Status.NOT_FOUND) {
      throw new FileNotFoundException(String.format("File spec %s not found", spec));
    } else if (matchResult.status() != Status.OK) {
      throw new IOException(
          String.format("Error matching file spec %s: status %s", spec, matchResult.status()));
    } else {
      List<Metadata> metadata = matchResult.metadata();
      if (metadata.size() != 1) {
        throw new IOException(
            String.format(
                "Expecting spec %s to match exactly one file, but matched %s: %s",
                spec, metadata.size(), metadata));
      }
      return metadata.get(0);
    }
  }

  /**
   * Returns {@link MatchResult MatchResults} for the given {@link ResourceId resourceIds}.
   *
   * @param resourceIds {@link ResourceId resourceIds} that might be derived from {@link #match},
   *     {@link ResourceId#resolve}, or {@link ResourceId#getCurrentDirectory()}.

View on GitHub (pinned to 12126d8942)

Solutions

1. Retry the match with exponential backoff for transient backend errors
2. Check filesystem credentials and permissions for the path's scheme
3. Log matchResult.status() to identify the provider-specific failure
4. Inspect the provider's logs (GCS client, HDFS client) for the root cause

Example fix

// before
Metadata md = FileSystems.matchSingleFileSpec(spec); // throws raw IOException
// after
try {
  Metadata md = FileSystems.matchSingleFileSpec(spec);
} catch (IOException e) {
  LOG.warn("Transient match failure for %s, retrying", spec, e);
  Metadata md = retryWithBackoff(() -> FileSystems.matchSingleFileSpec(spec));
}

// Check permissions/reachability of the scheme before matching
dryRunMatch(spec); // custom pre-check with limited retries and clear error reporting

try {
  return FileSystems.matchSingleFileSpec(spec);
} catch (IOException e) {
  if (attempt < MAX_RETRIES) return retryWithBackoff(spec, attempt + 1);
  throw new IOException("Match failed after retries for " + spec, e);
}

When it happens

Trigger: Calling FileSystems.matchSingleFileSpec when the filesystem backend returns MatchResult.Status.ERROR — e.g. GCS/S3 listing failures, permission issues surfaced by the provider, or transient network errors during match.

Common situations: GCS API throttling or transient 5xx during match; IAM/service-account permission failures on bucket listing; HDFS NameNode unavailability.

Understand the failure class

Background: 'Something went wrong' / 'Request failed (500)' / 'HTTP error! status: 404' — what failed HTTP requests actually mean and how to find the real cause — this error's family across 28 libraries.

Related errors

• File spec %s not found
• Failed to get metadata from MatchResult: %s.
• OffsetRetainer: failed to read offset from . Delete the file
• Un-globbable filesystem.
• Read-only filesystem.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/c5e38ee4736232c9. Report an issue: GitHub.