{"record":{"id":"d009280cebb47b23","repo":"apache/beam","slug":"expected-file-path-but-received-directory-path-s","errorCode":null,"errorMessage":"Expected file path but received directory path %s","messagePattern":"Expected file path but received directory path (.+?)","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/hadoop-file-system/src/main/java/org/apache/beam/sdk/io/hdfs/HadoopFileSystem.java","lineNumber":327,"sourceCode":"                \"Unable to create target directory %s. No further information provided by underlying filesystem.\",\n                targetDirectory));\n      }\n    }\n  }\n\n  @Override\n  protected void delete(Collection<HadoopResourceId> resourceIds) throws IOException {\n    for (HadoopResourceId resourceId : resourceIds) {\n      // ignore response as issues are surfaced with exception\n      final Path resourcePath = resourceId.toPath();\n      resourcePath.getFileSystem(configuration).delete(resourceId.toPath(), false);\n    }\n  }\n\n  @Override\n  protected HadoopResourceId matchNewResource(String singleResourceSpec, boolean isDirectory) {\n    if (singleResourceSpec.endsWith(\"/\") && !isDirectory) {\n      throw new IllegalArgumentException(\n          String.format(\"Expected file path but received directory path %s\", singleResourceSpec));\n    }\n    return !singleResourceSpec.endsWith(\"/\") && isDirectory\n        ? new HadoopResourceId(dropEmptyAuthority(singleResourceSpec + \"/\"))\n        : new HadoopResourceId(dropEmptyAuthority(singleResourceSpec));\n  }\n\n  @Override\n  protected String getScheme() {\n    return scheme;\n  }\n\n  @Override\n  protected void reportLineage(HadoopResourceId resourceId, Lineage lineage, LineageLevel level) {\n    URI uri = resourceId.toPath().toUri();\n    ImmutableList.Builder<String> segments = ImmutableList.builder();\n    if (uri.getAuthority() != null && !uri.getAuthority().isEmpty()) {\n      segments.add(uri.getAuthority());","sourceCodeStart":309,"sourceCodeEnd":345,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/hadoop-file-system/src/main/java/org/apache/beam/sdk/io/hdfs/HadoopFileSystem.java#L309-L345","documentation":"HadoopFileSystem.matchNewResource() rejects a resource spec that ends with '/' (a directory-style path) when the caller asserts it is a file via isDirectory=false. In Hadoop path conventions a trailing slash denotes a directory, so the assertion contradicts the spec.","triggerScenarios":"Calling HadoopFileSystem.matchNewResource(spec, false) — directly or via match/glob resolution — with singleResourceSpec ending in '/', or creating/reading a 'file' resource whose spec was built with a trailing slash.","commonSituations":"String-concatenating output file paths like base + \"/\" for filenames; path templates that already end with a slash; confusion between directory and file resource specs when resolving matches.","solutions":["Strip the trailing '/' from the spec before requesting a file resource.","Pass isDirectory=true if the path genuinely refers to a directory.","Build paths with a path-join utility rather than manual string concatenation."],"exampleFix":"// before\nmatchNewResource(\"hdfs://nn/data/output/\", false);\n// after\nmatchNewResource(\"hdfs://nn/data/output/part-0000.txt\", false);","handlingStrategy":"validation","validationCode":"if (singleResourceSpec.endsWith(\"/\") && !isDirectory) {\n  singleResourceSpec = singleResourceSpec.replaceAll(\"/+$\", \"\");\n}","typeGuard":"static boolean isFilePathSpec(String spec) { return !spec.endsWith(\"/\"); }","tryCatchPattern":"try {\n  resource = fileSystem.matchNewResource(spec, false);\n} catch (IllegalArgumentException e) {\n  resource = fileSystem.matchNewResource(spec.replaceAll(\"/+$\", \"\"), false);\n}","preventionTips":["Normalize paths: strip trailing '/' for files, add for directories","Use path-join helpers instead of string concatenation","Pass isDirectory consistently with the path's trailing slash"],"tags":["hadoop","hdfs","path","argument"],"backgroundTag":"invalid-url-format","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}