{"record":{"id":"771f9a8e915f5d5a","repo":"apache/hadoop","slug":"non-existing-file-path-create-option-is-not-s","errorCode":null,"errorMessage":"Non existing file: ${path}. Create option is not specified in ${flag}","messagePattern":"Non existing file: (.+?)\\. Create option is not specified in (.+?)","errorType":"exception","errorClass":"FileNotFoundException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CreateFlag.java","lineNumber":184,"sourceCode":"   * @param path Object representing the path; usually String or {@link Path}\n   * @param pathExists pass true if the path exists in the file system\n   * @param flag set of CreateFlag\n   * @throws IOException on error\n   * @throws HadoopIllegalArgumentException if the CreateFlag is invalid\n   */\n  public static void validate(Object path, boolean pathExists,\n      EnumSet<CreateFlag> flag) throws IOException {\n    validate(flag);\n    final boolean append = flag.contains(APPEND);\n    final boolean overwrite = flag.contains(OVERWRITE);\n    if (pathExists) {\n      if (!(append || overwrite)) {\n        throw new FileAlreadyExistsException(\"File already exists: \"\n            + path.toString()\n            + \". Append or overwrite option must be specified in \" + flag);\n      }\n    } else if (!flag.contains(CREATE)) {\n      throw new FileNotFoundException(\"Non existing file: \" + path.toString()\n          + \". Create option is not specified in \" + flag);\n    }\n  }\n\n  /**\n   * Validate the CreateFlag for the append operation. The flag must contain\n   * APPEND, and cannot contain OVERWRITE.\n   *\n   * @param flag enum set flag.\n   */\n  public static void validateForAppend(EnumSet<CreateFlag> flag) {\n    validate(flag);\n    if (!flag.contains(APPEND)) {\n      throw new HadoopIllegalArgumentException(flag\n          + \" does not contain APPEND\");\n    }\n  }\n}","sourceCodeStart":166,"sourceCodeEnd":202,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CreateFlag.java#L166-L202","documentation":"CreateFlag.validate(path, pathExists=false, flag) throws FileNotFoundException ('Non existing file: ... Create option is not specified') when the target does not exist and the flags lack CREATE. Effectively: you asked to append to (or otherwise touch) a file that is not there and did not authorize creating it.","triggerScenarios":"Calling create/append with EnumSet.of(CreateFlag.APPEND) on a path that does not exist (e.g., first run of a log writer), or any flag set without CREATE against a missing file - validated from fs.create(..., EnumSet, ...), fc.create, and append paths that call validate.","commonSituations":"Log/offset writers that append on every run but crash or get their first run against a fresh directory; typo'd or wrong-working-directory path resolving to a nonexistent file; a race where another process removed the file between the exists() check and the create call.","solutions":["Include CREATE together with APPEND if your filesystem honors create-on-append: EnumSet.of(CreateFlag.CREATE, CreateFlag.APPEND) (HDFS supports this: creates then appends).","Otherwise create explicitly first when missing: if (!fs.exists(f)) { try (out = fs.create(f)) {} } before the append.","Check path spelling and fs.getWorkingDirectory() - the exists()=false result is often a relative-path surprise."],"exampleFix":"// before\nEnumSet<CreateFlag> flags = EnumSet.of(CreateFlag.APPEND); // first run, file absent\nfs.create(path, perms, flags, 4096, (short)1, 1<<26, null); // FileNotFoundException\n\n// after\nEnumSet<CreateFlag> flags = fs.exists(path)\n    ? EnumSet.of(CreateFlag.APPEND)\n    : EnumSet.of(CreateFlag.CREATE, CreateFlag.APPEND);","handlingStrategy":"validation","validationCode":"if (!fs.exists(path) && !flags.contains(CreateFlag.CREATE)) {\n  flags = EnumSet.copyOf(flags);\n  flags.add(CreateFlag.CREATE); // or fail with a clear message\n}","typeGuard":null,"tryCatchPattern":"try {\n  out = fs.append(path);\n} catch (FileNotFoundException e) { // 'Non existing file... Create option is not specified'\n  // first run: create the file, then append next time (or use CREATE+APPEND where supported)\n}","preventionTips":["For append-style writers, handle the first run explicitly (create empty file at startup).","Use EnumSet.of(CREATE, APPEND) on filesystems that support create-on-append.","Log the resolved absolute path when the file is reported missing - it is often a cwd issue."],"tags":["hadoop","filesystem","file-not-found","create-flag","append"],"backgroundTag":"file-not-found-on-create","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}