{"record":{"id":"363ef43d9815fdc4","repo":"apache/hadoop","slug":"can-t-open-s-because-it-is-a-directory","errorCode":null,"errorMessage":"Can't open %s because it is a directory","messagePattern":"Can't open (.+?) because it is a directory","errorType":"exception","errorClass":"FileNotFoundException","httpStatus":null,"severity":"error","filePath":"hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/RawFileSystem.java","lineNumber":137,"sourceCode":"    return scheme;\n  }\n\n  @VisibleForTesting\n  String bucket() {\n    return bucket;\n  }\n\n  @Override\n  public void setConf(Configuration conf) {\n    super.setConf(conf);\n  }\n\n  @Override\n  public FSDataInputStream open(Path path, int bufferSize) throws IOException {\n    LOG.debug(\"Opening '{}' for reading.\", path);\n    RawFileStatus status = innerFileStatus(path);\n    if (status.isDirectory()) {\n      throw new FileNotFoundException(\n          String.format(\"Can't open %s because it is a directory\", path));\n    }\n\n    // Parse the range size from the hadoop conf.\n    long rangeSize = getConf().getLong(\n        ConfKeys.FS_OBJECT_STREAM_RANGE_SIZE,\n        ConfKeys.FS_OBJECT_STREAM_RANGE_SIZE_DEFAULT);\n    Preconditions.checkArgument(rangeSize > 0, \"Object storage range size must be positive.\");\n\n    FSInputStream fsIn = new ObjectMultiRangeInputStream(taskThreadPool, storage, path,\n        status.getLen(), rangeSize, status.checksum());\n    return new FSDataInputStream(fsIn);\n  }\n\n  public FSDataInputStream open(Path path, byte[] expectedChecksum, Range range) {\n    return new FSDataInputStream(\n        new ObjectRangeInputStream(storage, path, range, expectedChecksum));\n  }","sourceCodeStart":119,"sourceCodeEnd":155,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/RawFileSystem.java#L119-L155","documentation":"RawFileSystem.open stats the path first; when the target is a directory it throws FileNotFoundException with 'Can't open <path> because it is a directory'. Reading a prefix as a file has no meaning on object storage, so the open is refused before any range read starts.","triggerScenarios":"fs.open(dir) where dir exists only as a directory; passing a partition directory instead of files inside it; a missing glob so the input Path stays the directory itself.","commonSituations":"Input configuration points at the directory rather than files; a file reader API fed a directory path; job code that uses open() where the framework's input format was expected.","solutions":["Point open() at files: list the directory or glob with /dir/*","Guard with getFileStatus(path).isDirectory() and fail with an actionable message","Use the framework input format (FileInputFormat, Spark read) for directory inputs"],"exampleFix":"// before\nFSDataInputStream in = fs.open(new Path(\"/inputs/2024\")); // directory\n\n// after\nfor (FileStatus f : fs.listStatus(new Path(\"/inputs/2024\"))) {\n  if (f.isFile()) {\n    try (FSDataInputStream in = fs.open(f.getPath())) {\n      ... // read the file\n    }\n  }\n}","handlingStrategy":"validation","validationCode":"FileStatus st = fs.getFileStatus(p);\nif (st.isDirectory()) {\n  throw new IOException(\"Refusing to open a directory: \" + p);\n}","typeGuard":"static boolean isReadableFile(FileSystem fs, Path p) throws IOException {\n  return fs.exists(p) && fs.getFileStatus(p).isFile();\n}","tryCatchPattern":"try {\n  return fs.open(p);\n} catch (FileNotFoundException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"because it is a directory\")) {\n    // expand the directory: list and open files instead\n  } else {\n    throw e;\n  }\n}","preventionTips":["Validate isFile() before open() on user-supplied paths","Use globs or listStatus to expand directory inputs","Fail with actionable messages that distinguish 'is a directory' from 'missing'"],"tags":["hadoop","tos","open","directory","input-path"],"backgroundTag":"open-on-directory","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}