apache/hadoop · error · IllegalArgumentException

Error parsing libjars argument. Argument must be a valid URI

Error message

Error parsing libjars argument. Argument must be a valid URI: {}

What it means

Identical guard to the -files one, but for the -libjars option (conf 'tmpjars'): JobResourceUploader constructs java.net.URI from each libjar string before deciding whether to upload it or fetch it from the shared cache. An entry that is not a syntactically valid URI throws IllegalArgumentException('Error parsing libjars argument. Argument must be a valid URI: <entry>') with the URISyntaxException as cause.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobResourceUploader.java:297

  // Suppress warning for use of DistributedCache (it is everywhere).
  @SuppressWarnings("deprecation")
  @VisibleForTesting
  void uploadLibJars(Job job, Collection<String> libjars, Path submitJobDir,
      FsPermission mapredSysPerms, short submitReplication,
      Map<String, Boolean> fileSCUploadPolicies, Map<URI, FileStatus> statCache)
      throws IOException {
    Configuration conf = job.getConfiguration();
    Path libjarsDir = JobSubmissionFiles.getJobDistCacheLibjars(submitJobDir);
    if (!libjars.isEmpty()) {
      mkdirs(jtFs, libjarsDir, mapredSysPerms);
      Collection<URI> libjarURIs = new LinkedList<>();
      boolean foundFragment = false;
      for (String tmpjars : libjars) {
        URI tmpURI = null;
        try {
          tmpURI = new URI(tmpjars);
        } catch (URISyntaxException e) {
          throw new IllegalArgumentException("Error parsing libjars argument."
              + " Argument must be a valid URI: " + tmpjars, e);
        }
        Path tmp = new Path(tmpURI);
        URI newURI = null;
        boolean uploadToSharedCache = false;
        boolean fromSharedCache = false;
        if (scConfig.isSharedCacheLibjarsEnabled()) {
          newURI = useSharedCache(tmpURI, tmp.getName(), statCache, conf, true);
          if (newURI == null) {
            uploadToSharedCache = true;
          } else {
            fromSharedCache = true;
          }
        }

        if (newURI == null) {
          Path newPath =
              copyRemoteFiles(libjarsDir, tmp, conf, submitReplication);

View on GitHub (pinned to 2add963021)

Solutions

1. Encode the entry or use a proper file URI: -libjars file:///C:/libs/a.jar (forward slashes, no drive-colon ambiguity)
2. Pre-join local paths programmatically: conf.set("tmpjars", paths.stream().map(p -> new Path(p).toUri().toString()).collect(joining(",")))
3. Quote the whole -libjars value in the shell so spaces survive intact to the parser, then encode remaining illegal characters
4. Run a validation pass (new URI(s) per comma-separated token) before submit()

Example fix

# before
hadoop jar app.jar Driver -libjars "C:\lib\a.jar,C:\lib\b.jar" in out
# -> IllegalArgumentException: Error parsing libjars argument.

# after
hadoop jar app.jar Driver -libjars "file:///C:/lib/a.jar,file:///C:/lib/b.jar" in out

// validate/normalize every -libjars entry before submit
List<String> safe = new ArrayList<>();
for (String jar : StringUtils.getStrings(conf.get("tmpjars"))) {
  try {
    safe.add(new Path(jar).toUri().toString()); // encodes correctly
  } catch (Exception e) {
    throw new IllegalArgumentException("-libjars entry invalid: " + jar, e);
  }
}
conf.set("tmpjars", String.join(",", safe));

static boolean isValidCacheUri(String s) {
  try { new URI(s); return true; }
  catch (URISyntaxException e) { return false; }
}

When it happens

Trigger: -libjars with an unencoded space or a Windows backslash path (C:\m2\repo\...\a.jar); classpath strings assembled by build tools that embed 'C:' plus backslashes; multiple jars separated with the wrong delimiter or containing glob characters; entries passed through several layers of shell quoting that strip characters.

Common situations: Windows developer machines running hadoop/sqoop/pig with -libjars; Maven/Ivy-built command lines copied into scripts; spaces in repository paths; tools forwarding user-supplied paths unchecked.

Related errors

• Error parsing files argument. Argument must be a valid URI:
• Failed to create a URI (URISyntaxException) for the remote p
• Error parsing archives argument. Argument must be a valid UR
• Error parsing argument. Argument must be a valid URI: {}
• Error processing URI

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/997f4f26aa25ade4. Report an issue: GitHub.