{"record":{"id":"9bb29f9914efe1c7","repo":"apache/beam","slug":"failed-to-get-metadata-from-matchresult-s","errorCode":null,"errorMessage":"Failed to get metadata from MatchResult: %s.","messagePattern":"Failed to get metadata from MatchResult: (.+?)\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileSystems.java","lineNumber":373,"sourceCode":"\n    Collection<ResourceId> resourceIdsToDelete;\n    if (Sets.newHashSet(moveOptions)\n        .contains(MoveOptions.StandardMoveOptions.IGNORE_MISSING_FILES)) {\n      resourceIdsToDelete =\n          FluentIterable.from(matchResources(Lists.newArrayList(resourceIds)))\n              .filter(matchResult -> !matchResult.status().equals(Status.NOT_FOUND))\n              .transformAndConcat(\n                  new Function<MatchResult, Iterable<Metadata>>() {\n                    @SuppressFBWarnings(\n                        value = \"NP_METHOD_PARAMETER_TIGHTENS_ANNOTATION\",\n                        justification = \"https://github.com/google/guava/issues/920\")\n                    @Nonnull\n                    @Override\n                    public Iterable<Metadata> apply(@Nonnull MatchResult input) {\n                      try {\n                        return Lists.newArrayList(input.metadata());\n                      } catch (IOException e) {\n                        throw new RuntimeException(\n                            String.format(\"Failed to get metadata from MatchResult: %s.\", input),\n                            e);\n                      }\n                    }\n                  })\n              .transform(\n                  new Function<Metadata, ResourceId>() {\n                    @SuppressFBWarnings(\n                        value = \"NP_METHOD_PARAMETER_TIGHTENS_ANNOTATION\",\n                        justification = \"https://github.com/google/guava/issues/920\")\n                    @Nonnull\n                    @Override\n                    public ResourceId apply(@Nonnull Metadata input) {\n                      return input.resourceId();\n                    }\n                  })\n              .toList();\n    } else {","sourceCodeStart":355,"sourceCodeEnd":391,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileSystems.java#L355-L391","documentation":"Inside FileIO's ReadMatches/MatchAll expansion, converting a MatchResult to its Metadata iterable failed with an IOException; it is wrapped in a RuntimeException. This happens when the underlying filesystem cannot fetch metadata after an apparently successful match (e.g. expired listing result or backend error on metadata retrieval).","triggerScenarios":"Applying FileIO.match() followed by matchAll()/ReadMatches when input.metadata() on a MatchResult throws — typically transient backend failures (GCS/S3 errors, permissions changed between match and metadata fetch) or provider implementations that fetch metadata lazily.","commonSituations":"Race where files are deleted between match and metadata retrieval; transient cloud storage 5xx during pipeline execution; custom FileSystem implementations whose metadata() performs network I/O that fails.","solutions":["Enable runner retry: since this throws RuntimeException, the runner may retry the failed bundle — configure max retries","Check storage backend health/quotas and permissions for the matched paths","Reduce time between matching and reading to avoid races with file deletion","Catch and rethrow as a user-visible IOException by matching with FileSystems.match directly instead of relying on the PTransform"],"exampleFix":"// before\npipeline.apply(FileIO.match().filepattern(spec)).apply(FileIO.readMatches());\n// after\n// add retry/failure handling at pipeline level and monitor failed bundles\npipeline.apply(\"Match\", FileIO.match().filepattern(spec))\n       .apply(\"ReadMatches\", FileIO.readMatches());\n// and set runner retry options, e.g. DataflowPipelineOptions#setMaxNumWorkers / worker retry flags","handlingStrategy":"retry","validationCode":"// Pre-verify metadata is retrievable for matched files before the pipeline stage\nfor (Metadata m : FileSystems.match(spec).get(0).metadata()) {\n  FileSystems.matchSingleFileSpec(m.resourceId().toString()); // throws early if unreadable\n}","typeGuard":null,"tryCatchPattern":"try {\n  return matched.apply(FileIO.readMatches());\n} catch (RuntimeException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"Failed to get metadata from MatchResult\")) {\n    // let the runner retry the bundle; or re-match after backoff\n    throw e;\n  }\n  throw e;\n}","preventionTips":["Configure runner-level bundle retry for transient storage failures","Avoid deleting matched files before the pipeline reads them","Check storage quotas and rate limits to avoid throttling during matching","Use consistent, stable storage for intermediate matched paths"],"tags":["java","apache-beam","filesystem","runtime-exception","io"],"backgroundTag":"file-read-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}