{"record":{"id":"151d07da89e81de2","repo":"apache/hadoop","slug":"not-supported-151d07","errorCode":null,"errorMessage":"Not supported","messagePattern":"Not supported","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/RawFileSystem.java","lineNumber":232,"sourceCode":"\n  @Override\n  public FSDataOutputStream createNonRecursive(\n      Path path,\n      FsPermission permission,\n      EnumSet<CreateFlag> flag,\n      int bufferSize,\n      short replication,\n      long blockSize,\n      Progressable progress) throws IOException {\n    Path qualified = makeQualified(path);\n    return create(qualified, permission, flag.contains(CreateFlag.OVERWRITE),\n        bufferSize, replication, blockSize, progress);\n  }\n\n  @Override\n  public FSDataOutputStream append(Path f, int bufferSize, Progressable progress)\n      throws IOException {\n    throw new IOException(\"Not supported\");\n  }\n\n  /**\n   * Rename src path to dest path, if dest path is an existed dir,\n   * then FS will rename the src path under the dst dir.\n   * E.g. rename('/a/b', '/a/c') and dest 'c' is an existed dir,\n   * then the source path '/a/b' will be renamed with dest path '/a/b/c' internally.\n   *\n   * <ul>\n   *   <li>Return false if src doesn't exist</li>\n   *   <li>Return false if src is root</li>\n   *   <li>Return false if dst path is under src path, e.g. rename('/a/b', '/a/b/c')</li>\n   *   <li>Return false if dst path already exists</li>\n   *   <li>Return true if rename('/a/b', '/a/b') and 'b' is an existed file</li>\n   *   <li>Return true if rename('/a/b', '/a') and 'a' is an existed dir,\n   *   fs will rename '/a/b' to '/a/b' internally</li>\n   *   <li>Return false if rename('/a/b', '/a/b') and 'b' is an existed dir,\n   *   because fs will try to rename '/a/b' to '/a/b/b', which is under '/a/b', this behavior","sourceCodeStart":214,"sourceCodeEnd":250,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/RawFileSystem.java#L214-L250","documentation":"RawFileSystem.append() unconditionally throws IOException(\"Not supported\"). TOS (like S3 and most object stores) stores immutable objects: bytes cannot be appended to an existing object in place, so the Hadoop append API cannot be implemented. The connector fails fast rather than silently corrupting data.","triggerScenarios":"Calling fs.append(f, bufferSize, progress) on any tos:// path; frameworks that probe or require append (HBase WAL placement, some log writers, append-based streaming sinks) invoking append on the tos:// FileSystem.","commonSituations":"Pointing a workload that requires append (HBase, node logs, legacy FileOutputCommitter assumptions) at a tos:// root instead of HDFS; engines that try append-then-fallback when reopening an output file.","solutions":["Do not use append on tos://: write each chunk to a new object (unique part name) and merge/partition at read time","Move append-dependent components (e.g. HBase WAL) to HDFS and keep only bulk data on TOS","Probe capability before calling: fs.hasPathCapability(path, CommonPathCapabilities.APPEND) and skip the append path when false","For log aggregation, use timestamped/rotated object keys instead of one growing file"],"exampleFix":"// before\ntry (FSDataOutputStream out = fs.append(path, bufferSize, null)) { out.write(data); }\n\n// after: write a new object per chunk\nPath part = new Path(path.getParent(), path.getName() + \"-\" + UUID.randomUUID());\ntry (FSDataOutputStream out = fs.create(part, true)) { out.write(data); }","handlingStrategy":"validation","validationCode":"if (fs.hasPathCapability(path, \"append\")) {\n  out = fs.append(path, bufferSize, null);\n} else {\n  // tos://: rotate to a new object key per write\n}","typeGuard":null,"tryCatchPattern":"try { out = fs.append(path, bufferSize, null); }\ncatch (IOException e) {\n  if (\"Not supported\".equals(e.getMessage())) { /* rotate object keys instead */ }\n  else throw e;\n}","preventionTips":["Keep append-requiring engines (HBase WAL, append-based sinks) on HDFS","Design outputs as many immutable part files, merged at read","Probe hasPathCapability once at startup and disable append code paths"],"tags":["append","unsupported-operation","immutable-objects","object-storage","tos"],"backgroundTag":"unsupported-filesystem-operation","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}