apache/hadoop · error · IOException

Har: startLocalOutput not allowed

Error message

Har: startLocalOutput not allowed

What it means

startLocalOutput(Path fsOutputFile, Path tmpLocalFile) is the first half of FileSystem's local-staging commit protocol used by FileUtil.copy when copying INTO a filesystem: it returns a local temp path, the data is staged there, then completeLocalOutput commits it. HarFileSystem implements the protocol's members as unconditional throws, so any staged copy into a har fails here.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HarFileSystem.java:857

    throw new IOException("Har: copyfromlocalfile not allowed");
  }

  /**
   * copies the file in the har filesystem to a local file.
   */
  @Override
  public void copyToLocalFile(boolean delSrc, Path src, Path dst) 
    throws IOException {
    FileUtil.copy(this, src, getLocal(getConf()), dst, false, getConf());
  }
  
  /**
   * not implemented.
   */
  @Override
  public Path startLocalOutput(Path fsOutputFile, Path tmpLocalFile) 
    throws IOException {
    throw new IOException("Har: startLocalOutput not allowed");
  }
  
  /**
   * not implemented.
   */
  @Override
  public void completeLocalOutput(Path fsOutputFile, Path tmpLocalFile) 
    throws IOException {
    throw new IOException("Har: completeLocalOutput not allowed");
  }
  
  /**
   * not implemented.
   */
  @Override
  public void setOwner(Path p, String username, String groupname)
    throws IOException {
    throw new IOException("Har: setowner not allowed");

View on GitHub (pinned to 2add963021)

Solutions

1. Copy to the underlying HDFS with FileUtil.copy(local, src, hdfs, dst) and rebuild the archive afterwards
2. Route all 'into archive' flows through `hadoop archive`, which is the only supported writer
3. Add a destination-scheme guard before invoking FileUtil.copy

Example fix

// before
FileUtil.copy(localFs, src, harFs, new Path("har://hdfs-nn:8020/a/data.har/f"), false, conf); // hits startLocalOutput

// after
FileUtil.copy(localFs, src, hdfs, new Path("/a/data/f"), false, conf);
// then: hadoop archive -archiveName data.har -p /a/data /a

if ("har".equals(dst.toUri().getScheme())) {
  // FileUtil.copy's local-staging protocol cannot commit into a har
  throw new UnsupportedOperationException("staged copy into har is not supported: " + dst);
}

When it happens

Trigger: FileUtil.copy(localFs, src, harFs, dst, ...) or any code calling dstFs.startLocalOutput(harPath, tmp) — i.e., a copy-into-har that takes the local temp-file route rather than calling create directly.

Common situations: Reusing FileUtil.copy-based helpers with a har destination; porting local-to-remote copy code that relies on the staging protocol for large files.

Related errors

• Har: completeLocalOutput not allowed
• Har: create not allowed.
• Har: append not allowed.
• Har: setReplication not allowed
• Har: rename not allowed

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/8b10d5ab2bce718d. Report an issue: GitHub.