{"record":{"id":"2f86a5e8b31f2451","repo":"apache/hadoop","slug":"failed-to-create-a-uri-urisyntaxexception-for-th-2f86a5","errorCode":null,"errorMessage":"Failed to create a URI (URISyntaxException) for the remote path {}. This was based on the libjar parameter: {}","messagePattern":"Failed to create a URI \\(URISyntaxException\\) for the remote path (.+?)\\. This was based on the libjar parameter: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobResourceUploader.java","lineNumber":320,"sourceCode":"        boolean uploadToSharedCache = false;\n        boolean fromSharedCache = false;\n        if (scConfig.isSharedCacheLibjarsEnabled()) {\n          newURI = useSharedCache(tmpURI, tmp.getName(), statCache, conf, true);\n          if (newURI == null) {\n            uploadToSharedCache = true;\n          } else {\n            fromSharedCache = true;\n          }\n        }\n\n        if (newURI == null) {\n          Path newPath =\n              copyRemoteFiles(libjarsDir, tmp, conf, submitReplication);\n          try {\n            newURI = getPathURI(newPath, tmpURI.getFragment());\n          } catch (URISyntaxException ue) {\n            // should not throw a uri exception\n            throw new IOException(\n                \"Failed to create a URI (URISyntaxException) for the\"\n                    + \" remote path \" + newPath\n                    + \". This was based on the libjar parameter: \" + tmpjars,\n                ue);\n          }\n        }\n\n        if (!foundFragment) {\n          // We do not count shared cache paths containing fragments as a\n          // \"foundFragment.\" This is because these resources are not in the\n          // staging directory and will be added to the distributed cache\n          // separately.\n          foundFragment = (newURI.getFragment() != null) && !fromSharedCache;\n        }\n        Job.addFileToClassPath(new Path(newURI.getPath()), conf, jtFs, false);\n        if (fromSharedCache) {\n          // We simply add this URI to the distributed cache. It will not come\n          // from the staging directory (it is in the shared cache), so we","sourceCodeStart":302,"sourceCodeEnd":338,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobResourceUploader.java#L302-L338","documentation":"The libjars counterpart of the defensive post-copy check: after copyRemoteFiles places the jar in the staging libjars directory, getPathURI(newPath, tmpURI.getFragment()) rebuilds the final URI. The URI constructor throws URISyntaxException when the fragment taken from your original -libjars entry is malformed, and the code wraps it as IOException('Failed to create a URI (URISyntaxException) ... based on the libjar parameter'). The comment says it 'should not throw', so this indicates a bad '#alias' fragment, not a copy failure.","triggerScenarios":"-libjars 'hdfs://nn/l/a.jar#dep jar' (space in the fragment alias); fragments with reserved characters ('?', backslash, quotes); an entry whose fragment came from an unescaped shell variable.","commonSituations":"Adding mnemonic link names to dependency jars; CI pipelines interpolating version strings containing '+' or spaces into the alias; documentation examples that show '#name' without restricting the character set.","solutions":["Use identifier-safe aliases: 'a.jar#depjar' instead of 'a.jar#dep jar'","Percent-encode spaces in aliases as %20","Omit the fragment when a custom name is unnecessary — the file name is used as the link name","Pre-check: new URI(null, null, null, fragment) must not throw before you submit"],"exampleFix":"# before\nhadoop jar app.jar Driver -libjars \"hdfs://nn/libs/guava.jar#guava lib\" in out\n# IOException: Failed to create a URI (URISyntaxException) ... libjar parameter\n\n# after\nhadoop jar app.jar Driver -libjars \"hdfs://nn/libs/guava.jar#guava-lib\" in out","handlingStrategy":"validation","validationCode":"// reject libjar entries with unsafe '#alias' fragments\nfor (String jar : StringUtils.getStrings(conf.get(\"tmpjars\"))) {\n  int i = jar.indexOf('#');\n  if (i >= 0) {\n    try { new URI(null, null, null, jar.substring(i + 1)); }\n    catch (URISyntaxException e) {\n      throw new IllegalArgumentException(\"Bad alias in -libjars entry: \" + jar, e);\n    }\n  }\n}","typeGuard":"static boolean hasSafeFragment(String entry) {\n  int i = entry.indexOf('#');\n  if (i < 0) return true;\n  try { new URI(null, null, null, entry.substring(i + 1)); return true; }\n  catch (URISyntaxException e) { return false; }\n}","tryCatchPattern":null,"preventionTips":["Use identifier-style aliases for libjars (depjar, not 'dep jar')","Encode spaces as %20 when an alias must contain one","Default to no fragment — the jar file name is usually sufficient"],"tags":["distributed-cache","uri","fragment","libjars","mapreduce"],"backgroundTag":"uri-parse-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}