{"record":{"id":"5bd8fe1da418f208","repo":"apache/beam","slug":"failed-to-validate-s","errorCode":null,"errorMessage":"Failed to validate %s","messagePattern":"Failed to validate (.+?)","errorType":"validation","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"sdks/java/core/src/main/java/org/apache/beam/sdk/io/TFRecordIO.java","lineNumber":196,"sourceCode":"    }\n\n    @Override\n    public PCollection<byte[]> expand(PBegin input) {\n      if (getFilepattern() == null) {\n        throw new IllegalStateException(\n            \"Need to set the filepattern of a TFRecordIO.Read transform\");\n      }\n\n      if (getValidate()) {\n        checkState(getFilepattern().isAccessible(), \"Cannot validate with a RVP.\");\n        try {\n          MatchResult matches = FileSystems.match(getFilepattern().get());\n          checkState(\n              !matches.metadata().isEmpty(),\n              \"Unable to find any files matching %s\",\n              getFilepattern().get());\n        } catch (IOException e) {\n          throw new IllegalStateException(\n              String.format(\"Failed to validate %s\", getFilepattern().get()), e);\n        }\n      }\n\n      return input.apply(\"Read\", org.apache.beam.sdk.io.Read.from(getSource()));\n    }\n\n    // Helper to create a source specific to the requested compression type.\n    protected FileBasedSource<byte[]> getSource() {\n      return CompressedSource.from(new TFRecordSource(getFilepattern()))\n          .withCompression(getCompression());\n    }\n\n    @Override\n    public void populateDisplayData(DisplayData.Builder builder) {\n      super.populateDisplayData(builder);\n      builder\n          .add(","sourceCodeStart":178,"sourceCodeEnd":214,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/core/src/main/java/org/apache/beam/sdk/io/TFRecordIO.java#L178-L214","documentation":"When TFRecordIO.Read has validate=true (the default), expand() calls FileSystems.match() on the filepattern; if matching raises an IOException (filesystem unreachable, bad scheme, credentials problem), it wraps it in this IllegalStateException. It signals that the input files could not be verified to exist before the pipeline runs.","triggerScenarios":"TFRecordIO.read().from(pattern) with validate enabled and FileSystems.match throwing IOException — wrong filesystem scheme, no filesystem registered for the scheme, or network/auth failure accessing GCS/S3/local path.","commonSituations":"Typo in gs:// or s3:// scheme; missing Hadoop/GCS connector on classpath; expired credentials; reading a local path that doesn't resolve.","solutions":["Fix the filepattern so it matches an accessible location and check the wrapped IOException cause for the root reason.","Verify a FileSystem for the URI scheme is registered (add beam-sdks-java-io-... dependency for GCS/S3).","Set .withValidate(false) only if you accept deferring existence checks to runtime."],"exampleFix":"// before\nTFRecordIO.read().from(\"gcs://wrong-scheme/data/*.tfrecord\");\n\n// after\nTFRecordIO.read().from(\"gs://my-bucket/data/*.tfrecord\");","handlingStrategy":"validation","validationCode":"MatchResult mr = FileSystems.match(filePattern);\nif (mr.status() == MatchResult.Status.NOT_FOUND || mr.metadata().isEmpty()) {\n  throw new IllegalArgumentException(\"No files match: \" + filePattern);\n}","typeGuard":null,"tryCatchPattern":"try { ... } catch (IllegalStateException e) { if (e.getCause() instanceof IOException) { log.error(\"TFRecord validation IO failure for pattern\", e.getCause()); } throw e; }","preventionTips":["Check filesystem scheme and credentials before submitting the pipeline.","Keep validate enabled in staging to catch bad patterns early."],"tags":["java","beam","tfrecord","validation","filesystem"],"backgroundTag":"file-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}