apache/hadoop · error · IllegalArgumentException

Could not determine filetype for: {path}

Error message

Could not determine filetype for: {path}

What it means

HttpFSFileSystem.FILE_TYPE.getType (HttpFSFileSystem.java:180-191) maps a Hadoop FileStatus to the JSON type field of the WebHDFS/HttpFS response by probing isFile(), then isDirectory(), then isSymlink(); if all three return false it throws this IllegalArgumentException. Hadoop's own FileStatus always satisfies one of them, so in practice this fires for custom or mocked FileStatus implementations behind HttpFS (e.g. third-party FileSystem adapters or test doubles) that leave the object in an inconsistent type state.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/client/HttpFSFileSystem.java:187

  public static final String UPLOAD_CONTENT_TYPE= "application/octet-stream";

  public static final String SNAPSHOT_JSON = "Path";

  public enum FILE_TYPE {
    FILE, DIRECTORY, SYMLINK;

    public static FILE_TYPE getType(FileStatus fileStatus) {
      if (fileStatus.isFile()) {
        return FILE;
      }
      if (fileStatus.isDirectory()) {
        return DIRECTORY;
      }
      if (fileStatus.isSymlink()) {
        return SYMLINK;
      }
      throw new IllegalArgumentException("Could not determine filetype for: " +
                                         fileStatus.getPath());
    }
  }

  public static final String FILE_STATUSES_JSON = "FileStatuses";
  public static final String FILE_STATUS_JSON = "FileStatus";
  public static final String FS_STATUS_JSON = "FsStatus";
  public static final String PATH_SUFFIX_JSON = "pathSuffix";
  public static final String TYPE_JSON = "type";
  public static final String LENGTH_JSON = "length";
  public static final String OWNER_JSON = "owner";
  public static final String GROUP_JSON = "group";
  public static final String PERMISSION_JSON = "permission";
  public static final String ACCESS_TIME_JSON = "accessTime";
  public static final String MODIFICATION_TIME_JSON = "modificationTime";
  public static final String BLOCK_SIZE_JSON = "blockSize";
  public static final String CHILDREN_NUM_JSON = "childrenNum";
  public static final String FILE_ID_JSON = "fileId";

View on GitHub (pinned to 2add963021)

Solutions

1. Fix the FileStatus producer: construct it with an explicit type, e.g. new FileStatus(len, isDir, repl, bs, mtime, atime, permission, owner, group, path) with isDir set correctly, or call setIsDir/setSymlink on mutable implementations.
2. In tests, build FileStatus with real constructors instead of default Mockito mocks, or stub isFile()/isDirectory() to return true for one.
3. If the status is legitimately unknown, resolve it before serialization rather than passing a neutral FileStatus down.

Example fix

// before
FileStatus st = mock(FileStatus.class);          // all probes false -> 3618
// after
FileStatus st = new FileStatus(1024, false, 1, 128L<<20,
    System.currentTimeMillis(), System.currentTimeMillis(),
    FsPermission.getDefault(), "owner", "group", new Path("/f"));
// (isDir=false => isFile() true)

import org.apache.hadoop.fs.FileStatus;
static String fileTypeOf(FileStatus st) {
  if (st.isFile()) return "FILE";
  if (st.isDirectory()) return "DIRECTORY";
  if (st.isSymlink()) return "SYMLINK";
  throw new IllegalStateException("FileStatus has no type: " + st.getPath());
}

static boolean hasKnownFileType(FileStatus st) {
  return st.isFile() || st.isDirectory() || st.isSymlink();
}

try {
  return FILE_TYPE.getType(status).toString();
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("Could not determine filetype")) {
    log.error("backing FileSystem returned typeless FileStatus for {}", status.getPath());
  }
  throw e;
}

When it happens

Trigger: An HttpFS server backed by a custom FileSystem whose FileStatus sets neither the file nor directory flag (e.g. constructed via a bare new FileStatus() with nothing set and isSymlink() overridden to false); unit/integration tests feeding hand-built FileStatus objects into code that serializes them via FILE_TYPE.getType; FilterFileSystem shims that forward to a partially implemented backing store.

Common situations: Writing a custom FileSystem to expose a store through HttpFS and forgetting to mark statuses as file or directory; mock-based tests where Mockito returns default false for all three probes; upgrades where a third-party connector lags the FileStatus API (e.g. symlink support stubs returning false).

Related errors

• %s not found: %s
• Implementation Error: {getClass()} that threw an UnresolvedL
• Can't read FileStatusProto with negative size of ${size}
• No Path in deserialized FileStatus
• No type in deserialized FileStatus

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/311b0e4ceaf53725. Report an issue: GitHub.