{"record":{"id":"56a52a687a946172","repo":"apache/hadoop","slug":"pathnotfound","errorCode":"PathNotFound","errorMessage":"openFileForWrite must be used with files and not directories","messagePattern":"openFileForWrite must be used with files and not directories","errorType":"exception","errorClass":"AbfsRestOperationException","httpStatus":404,"severity":"error","filePath":"hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java","lineNumber":1107,"sourceCode":"\n  public OutputStream openFileForWrite(final Path path,\n      final FileSystem.Statistics statistics, final boolean overwrite,\n      TracingContext tracingContext) throws IOException {\n    try (AbfsPerfInfo perfInfo = startTracking(\"openFileForWrite\", \"getPathStatus\")) {\n      LOG.debug(\"openFileForWrite filesystem: {} path: {} overwrite: {}\",\n              getClient().getFileSystem(),\n              path,\n              overwrite);\n\n      String relativePath = getRelativePath(path);\n      AbfsClient writeClient = getClientHandler().getIngressClient();\n\n      final AbfsRestOperation op = getClient()\n          .getPathStatus(relativePath, false, tracingContext, null);\n      perfInfo.registerResult(op.getResult());\n\n      if (getClient().checkIsDir(op.getResult())) {\n        throw new AbfsRestOperationException(\n              AzureServiceErrorCode.PATH_NOT_FOUND.getStatusCode(),\n              AzureServiceErrorCode.PATH_NOT_FOUND.getErrorCode(),\n              \"openFileForWrite must be used with files and not directories\",\n              null);\n      }\n\n      final long contentLength = extractContentLength(op.getResult());\n      final long offset = overwrite ? 0 : contentLength;\n\n      perfInfo.registerSuccess(true);\n\n      boolean isAppendBlob = false;\n      if (isAppendBlobKey(path.toString())) {\n        isAppendBlob = true;\n      }\n\n      AbfsLease lease = maybeCreateLease(relativePath, tracingContext);\n      final String eTag = extractEtagHeader(op.getResult());","sourceCodeStart":1089,"sourceCodeEnd":1125,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java#L1089-L1125","documentation":"openFileForWrite backs FS.create(), FS.append(), and FS.openFile(...WRITE). It first issues GetPathStatus; if the existing path is a directory it aborts with an AbfsRestOperationException whose status code and error code are borrowed from AzureServiceErrorCode.PATH_NOT_FOUND (HTTP 404, PathNotFound). The 404 code is misleading - the path exists, it is just the wrong resource type, as the message says.","triggerScenarios":"fs.create(path, overwrite) or fs.append(path), or openFile with WRITE/CREATE options, on a path that currently exists as a directory (previously created by mkdir). Races where another actor mkdirs the path between check and write also produce it.","commonSituations":"Output committers/partition writers colliding with partition directory names (e.g. trying to write file 'date=2024' when 'date=2024/' was created as a directory); re-running jobs that created directories on prior attempts; path-building bugs appending '/' or reusing directory names as file names.","solutions":["Remove or rename the conflicting directory before creating the file: fs.delete(dirPath, true) or rename it, then retry create.","Pre-check with fs.getFileStatus(path).isFile() when the path is expected to exist, and fail with a clear upstream message.","Fix caller-side path construction: no trailing slash on file paths, distinct names for directories and files at the same level."],"exampleFix":"// before\ntry (FSDataOutputStream out = fs.create outputPath) { ... }\n\n// after\nif (fs.exists(outputPath) && !fs.getFileStatus(outputPath).isFile()) {\n  throw new IOException(\"Output path is a directory: \" + outputPath);\n}\ntry (FSDataOutputStream out = fs.create(outputPath, true)) { ... }","handlingStrategy":"validation","validationCode":"if (fs.exists(path)) {\n  FileStatus st = fs.getFileStatus(path);\n  if (st.isDirectory()) {\n    throw new IOException(\"Refusing to open for write, path is a directory: \" + path);\n  }\n}","typeGuard":null,"tryCatchPattern":"try { out = fs.create(path, overwrite); } catch (AbfsRestOperationException e) {\n  if (\"PathNotFound\".equals(e.getErrorCode()) && fs.exists(path) && fs.getFileStatus(path).isDirectory()) {\n    // real cause: directory at file path - not a 404\n  }\n}","preventionTips":["Expect PATH_NOT_FOUND (404) on this error even though the path exists.","Separate directory names and file names in generated layouts.","Guard create/append on existing paths with an isFile() check in job boilerplate."],"tags":["azure","abfs","create","append","path-conflict","is-a-directory"],"backgroundTag":"write-to-directory","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}