apache/hadoop · error · IOException

Cannot clone: stream belongs to a different DFSClient instan

Error message

Cannot clone: stream belongs to a different DFSClient instance

What it means

Error "Cannot clone: stream belongs to a different DFSClient instance" thrown in apache/hadoop.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java:456

   * @throws IOException if the stream cannot be cloned
   */
  public FSDataInputStream cloneDataInputStream(FSDataInputStream existing)
      throws IOException {
    statistics.incrementReadOps(1);
    storageStatistics.incrementOpCounter(OpType.OPEN);
    InputStream wrapped = existing.getWrappedStream();
    DFSInputStream dfsis;
    if (wrapped instanceof DFSInputStream dfsIn) {
      dfsis = dfsIn;
    } else if (wrapped instanceof
        org.apache.hadoop.crypto.CryptoInputStream cryptoIn) {
      dfsis = (DFSInputStream) cryptoIn.getWrappedStream();
    } else {
      throw new IOException("Cannot clone: underlying stream is "
          + wrapped.getClass().getName() + ", not a DFSInputStream");
    }
    if (dfsis.getDFSClient() != dfs) {
      throw new IOException(
          "Cannot clone: stream belongs to a different DFSClient instance");
    }
    LocatedBlocks locatedBlocks = dfsis.getLocatedBlocks();
    String src = dfsis.getSrc();
    DFSInputStream clone = dfs.open(src,
        dfs.getConf().getIoBufferSize(), verifyChecksum, locatedBlocks);
    try {
      return dfs.createWrappedInputStream(clone);
    } catch (IOException e) {
      clone.close();
      throw e;
    }
  }

  @Override
  public String getErasureCodingPolicyName(FileStatus fileStatus) {
    if (!(fileStatus instanceof HdfsFileStatus)) {
      return null;

View on GitHub (pinned to 2add963021)

Solutions

1. Clone the stream using the same DFSClient/FileSystem instance that opened it.
2. Reopen the file with the current FileSystem instead of cloning a stream from another client.

When it happens

Trigger: An open-stream clone is requested for a stream that was created by a different DFSClient instance; clones must share the same client.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/088b9269cf82e16e. Report an issue: GitHub.