apache/hadoop · error · IllegalStateException

Cannot parse URI %s

Error message

Cannot parse URI %s

What it means

IllegalStateException from SinglePendingCommit.destinationPath() when the record's 'uri' text cannot be parsed by java.net.URI. Each SinglePendingCommit in a pendingset-*.json file stores the destination object URI as a string; destinationPath() first asserts the uri is non-empty (Guava 'Empty uri' check) and then converts it to a Hadoop Path. Malformed URIs (unencoded spaces, bad characters, relative references) throw URISyntaxException, which is rethrown as this IllegalStateException. Note the cause is not chained, so only the message shows the offending uri.

Source

Thrown at hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/commit/files/SinglePendingCommit.java:300

  @Override
  public IOStatistics save(final FileSystem fs,
      final Path path,
      final JsonSerialization<SinglePendingCommit> serializer) throws IOException {
    return saveFile(fs, path, this, serializer, true);
  }

  /**
   * Build the destination path of the object.
   * @return the path
   * @throws IllegalStateException if the URI is invalid
   */
  public Path destinationPath() {
    Preconditions.checkState(StringUtils.isNotEmpty(uri), "Empty uri");
    try {
      return new Path(new URI(uri));
    } catch (URISyntaxException e) {
      throw new IllegalStateException("Cannot parse URI " + uri);
    }
  }

  /**
   * Get the number of etags.
   * @return the size of the etag list.
   */
  public int getPartCount() {
    return etags.size();
  }

  /**
   * Iterate over the etags.
   * @return an iterator.
   */
  @Override
  public Iterator<UploadEtag> iterator() {
    return etags.iterator();

View on GitHub (pinned to 2add963021)

Solutions

1. Inspect the uri field of the pendingset-*.json files under the job's staging/magic output directory to find the malformed entry
2. Abort the job commit (committer cleanup / delete the output path) and rerun the job so clean pendingsets are produced
3. Ensure destPath passed to uploadFileToPendingCommit is a fully qualified s3a:// path (Path.makeQualified) with URI-safe encoding
4. If the JSON must be repaired manually, percent-encode unsafe characters in the uri field

Example fix

// before
Path dest = new Path("bucket/out/part-00000"); // relative, unqualified

// after
Path dest = new Path("s3a://bucket/out/part-00000").makeQualified(fs.getUri(), fs.getWorkingDirectory());

for (SinglePendingCommit c : pendingSet.commits()) {
  try {
    new java.net.URI(c.uri()); // cheap pre-parse
  } catch (java.net.URISyntaxException bad) {
    LOG.error("Bad pending commit uri: {}", c.uri());
    return false; // refuse to commit this pendingset
  }
}
return true;

try {
  Path dest = commit.destinationPath();
} catch (IllegalStateException e) {
  // message contains the offending uri; the URISyntaxException cause is NOT chained
  throw new IOException("Corrupt pending commit record: " + e.getMessage(), e);
}

When it happens

Trigger: Committing a job whose pending commit records contain a uri like 's3a://bucket/my file' (unencoded space) or 'bucket/key' (no scheme); destinationPath() is called while loading pendingsets during commitJob or in tooling that audits pending commits.

Common situations: The staging/magic task wrote a destPath that was not fully qualified or not URI-encoded; the pendingset JSON was hand-edited or corrupted; versions of the writer and reader disagree on how the uri field was serialized.

Related errors

• Multipart IO request {sdkRequest} rejected {header}
• Multipart uploads are disabled for the FileSystem, the commi
• Unable to recover task %s
• Task attempt {attemptID} has a self-generated job UUID
• Mismatch in Job ID (%s) and commit job ID (%s)

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/6e50e7091c8bc348. Report an issue: GitHub.