{"record":{"id":"c1d46216b8d3e277","repo":"apache/hadoop","slug":"invalid-path-uri-cannot-contain-both-a-uri-f-c1d462","errorCode":null,"errorMessage":"Invalid path URI: {} - cannot contain both a URI fragment and a wildcard","messagePattern":"Invalid path URI: (.+?) - cannot contain both a URI fragment and a wildcard","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java","lineNumber":320,"sourceCode":"      HashMap<Path, String> linkLookup = new HashMap<Path, String>();\n      if (withLinks != null) {\n        for (URI u: withLinks) {\n          Path p = new Path(u);\n          FileSystem remoteFS = p.getFileSystem(conf);\n          String name = p.getName();\n          String wildcard = null;\n\n          // If the path is wildcarded, resolve its parent directory instead\n          if (name.equals(DistributedCache.WILDCARD)) {\n            wildcard = name;\n            p = p.getParent();\n          }\n\n          p = remoteFS.resolvePath(p.makeQualified(remoteFS.getUri(),\n              remoteFS.getWorkingDirectory()));\n\n          if ((wildcard != null) && (u.getFragment() != null)) {\n            throw new IOException(\"Invalid path URI: \" + p + \" - cannot \"\n                + \"contain both a URI fragment and a wildcard\");\n          } else if (wildcard != null) {\n            name = p.getName() + Path.SEPARATOR + wildcard;\n          } else if (u.getFragment() != null) {\n            name = u.getFragment();\n          }\n\n          // If it's not a JAR, add it to the link lookup.\n          if (!StringUtils.toLowerCase(name).endsWith(\".jar\")) {\n            String old = linkLookup.put(p, name);\n\n            if ((old != null) && !name.equals(old)) {\n              LOG.warn(\"The same path is included more than once \"\n                  + \"with different links or wildcards: \" + p + \" [\" +\n                  name + \", \" + old + \"]\");\n            }\n          }\n        }","sourceCodeStart":302,"sourceCodeEnd":338,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java#L302-L338","documentation":"While building LocalResources from cache URIs, MRApps resolves each path (resolving the parent when a '*' wildcard is present) and then applies the same mutual-exclusion rule as the submitter path: a URI may carry a wildcard or a fragment, not both, throwing IOException('Invalid path URI: ... cannot contain both a URI fragment and a wildcard'). With a wildcard, link names come from the resolved directory listing; a fragment would try to name one link for many files.","triggerScenarios":"A job conf cache entry such as 'hdfs://nn/libs/*#libs' processed during job setup/AM localization; archives with the same combination ('hdfs://nn/arch/*#a').","commonSituations":"Config templates that append '#name' to every cache/archive path including wildcarded ones; user attempts to control the symlink name of a directory of jars; migration scripts combining two previously-separate patterns.","solutions":["Remove the fragment from wildcard entries: 'hdfs://nn/libs/*' localizes every file with its own name","Or expand the wildcard yourself and list explicit files, each optionally with its own fragment","Sanitize cache URI configs to reject entries containing both '*' and '#' before submission"],"exampleFix":"# before\njob.addCacheFile(new URI(\"hdfs://nn/libs/*#libs\")); // throws IOException at setup\n\n# after\njob.addCacheFile(new URI(\"hdfs://nn/libs/*\")); // every jar linked under its own name\n// or list files explicitly if a specific link name is needed:\njob.addCacheFile(new URI(\"hdfs://nn/libs/a.jar#a.jar\"));","handlingStrategy":"validation","validationCode":"static boolean hasWildcardFragmentConflict(URI u) {\n  Path p = new Path(u);\n  return DistributedCache.WILDCARD.equals(p.getName()) && u.getFragment() != null;\n}\n// strip the fragment from wildcarded entries before conf reaches MR setup","typeGuard":null,"tryCatchPattern":"try {\n  MRApps.setupDistributedCache(conf, localResources);\n} catch (IOException e) {\n  if (e.getMessage().contains(\"fragment and a wildcard\")) {\n    // rewrite the offending cache URI without the fragment and retry setup once\n  }\n  throw e;\n}","preventionTips":["One mechanism per entry: '*' for whole directories, '#name' for single files","Run the same fragment/wildcard validation in both submitter and AM code paths to fail early","Automate cache-config generation instead of string-concatenating fragments onto paths"],"tags":["hadoop","mapreduce","distributed-cache","wildcard","uri","fragment"],"backgroundTag":"invalid-distributed-cache-uri","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}