{"record":{"id":"3d9e03f9588c4d95","repo":"apache/hadoop","slug":"append-is-not-supported-by-baidubosfilesystem","errorCode":null,"errorMessage":"Append is not supported by BaiduBosFileSystem","messagePattern":"Append is not supported by BaiduBosFileSystem","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"hadoop-cloud-storage-project/hadoop-bos/src/main/java/org/apache/hadoop/fs/bos/BaiduBosFileSystem.java","lineNumber":151,"sourceCode":"    if (path.isAbsolute()) {\n      return path;\n    }\n    return new Path(workingDir, path);\n  }\n\n  /**\n   * This optional operation is not yet supported.\n   *\n   * @param f the file to append to\n   * @param bufferSize the buffer size\n   * @param progress for reporting progress\n   * @return never returns\n   * @throws IOException always thrown\n   */\n  @Override\n  public FSDataOutputStream append(Path f, int bufferSize,\n      Progressable progress) throws IOException {\n    throw new UnsupportedOperationException(\n        \"Append is not supported by BaiduBosFileSystem\");\n  }\n\n  @Override\n  public RemoteIterator<LocatedFileStatus> listLocatedStatus(\n      final Path f, final PathFilter filter) throws IOException {\n    return new RemoteIterator<LocatedFileStatus>() {\n      private final FileStatus[] stats = listStatus(f, filter);\n      private int i = 0;\n\n      public boolean hasNext() {\n        return i < stats.length;\n      }\n\n      public LocatedFileStatus next() throws IOException {\n        if (!hasNext()) {\n          throw new NoSuchElementException(\n              \"No more entry in \" + f);","sourceCodeStart":133,"sourceCodeEnd":169,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-cloud-storage-project/hadoop-bos/src/main/java/org/apache/hadoop/fs/bos/BaiduBosFileSystem.java#L133-L169","documentation":"StreamUtil's class-location logic resolves where a class was loaded from (jar path or class directory) so streaming can ship that code to task nodes. After normalizing the resource URL it strips the trailing '/my/package/Class.class' by searching for the relative class path; if lastIndexOf(relPath) returns -1 it throws IllegalArgumentException 'invalid codePath'. This means the classloader produced a URL that does not end with the package-relative path of the class, so the jar/root cannot be derived.","triggerScenarios":"Calling StreamUtil.stripClass(className) (used when auto-shipping -mapper/-reducer Java classes) when the class resource URL has been rewritten by a non-standard classloader — e.g. agents, proxies (Hibernate/CGLIB style), Spring boot executable-jar loaders, or URLs with query/fragment suffixes so 'package/Class.class' no longer literally matches.","commonSituations":"Running streaming with a Java mapper/reducer inside an application container or with -libjars plus a custom ClassLoader; classes generated at runtime; shaded/renamed packages where the .class resource path differs from the Class.getName(); JPMS/module classpath URLs.","solutions":["Avoid the auto-detection path: package your mapper in a jar and register it explicitly with -libjars / job.setJar(...), or set mapreduce.job.jar, so the streaming code path shipping is skipped.","If you control the launcher, ensure the class is loaded from a plain file: or jar: URL whose path literally ends with <package-path>/<Class>.class (no custom URL schemes).","Check for runtime-generated or proxied classes being passed as the command class and pass the real class name instead.","Upgrade/verify you are not hitting a classloader rewrite from an agent (e.g. coverage or APM agents prepending transformed class locations)."],"exampleFix":"# before: rely on auto-detection of class location\nhadoop jar hadoop-streaming.jar -mapper com.myco.MyMapper -reducer ...\n\n# after: ship the jar explicitly, bypassing class-location detection\nhadoop jar hadoop-streaming.jar -libjars my-mapper.jar \\\n  -mapper com.myco.MyMapper -input ... -output ...","handlingStrategy":"try-catch","validationCode":"java.net.URL u = loader.getResource(className.replace('.', '/') + \".class\");\nif (u == null || !u.toString().contains(className.replace('.', '/') + \".class\")) {\n  // auto-detection will fail; fall back to explicit jar shipping\n  job.setJar(\"my-mapper.jar\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  String jar = StreamUtil.stripClass(mapperClassName);\n} catch (IllegalArgumentException e) {\n  // classloader URL not strip-able: ship jar explicitly instead of guessing\n  jobConf.setJarByClass(MapperClass.class);\n}","preventionTips":["Prefer explicit -libjars / setJar over relying on class-location auto-detection.","Test streaming jobs under the same classloader setup used in production (agents, containers).","Avoid runtime-generated/proxy classes as streaming commands."],"tags":["hadoop","streaming","classpath","classloader","illegal-argument"],"backgroundTag":"classpath-resolution-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}