apache/hadoop · error · IOException

Cannot clone: underlying stream is, not a DFSInputStream

Error message

Cannot clone: underlying stream is, not a DFSInputStream

What it means

Error "Cannot clone: underlying stream is, not a DFSInputStream" thrown in apache/hadoop.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java:452

   * but shares the same block location metadata.
   *
   * @param existing an open input stream obtained from this filesystem
   * @return a new independent input stream for the same file
   * @throws IOException if the stream cannot be cloned
   */
  public FSDataInputStream cloneDataInputStream(FSDataInputStream existing)
      throws IOException {
    statistics.incrementReadOps(1);
    storageStatistics.incrementOpCounter(OpType.OPEN);
    InputStream wrapped = existing.getWrappedStream();
    DFSInputStream dfsis;
    if (wrapped instanceof DFSInputStream dfsIn) {
      dfsis = dfsIn;
    } else if (wrapped instanceof
        org.apache.hadoop.crypto.CryptoInputStream cryptoIn) {
      dfsis = (DFSInputStream) cryptoIn.getWrappedStream();
    } else {
      throw new IOException("Cannot clone: underlying stream is "
          + wrapped.getClass().getName() + ", not a DFSInputStream");
    }
    if (dfsis.getDFSClient() != dfs) {
      throw new IOException(
          "Cannot clone: stream belongs to a different DFSClient instance");
    }
    LocatedBlocks locatedBlocks = dfsis.getLocatedBlocks();
    String src = dfsis.getSrc();
    DFSInputStream clone = dfs.open(src,
        dfs.getConf().getIoBufferSize(), verifyChecksum, locatedBlocks);
    try {
      return dfs.createWrappedInputStream(clone);
    } catch (IOException e) {
      clone.close();
      throw e;
    }
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Only clone streams opened via DistributedFileSystem whose underlying stream is a DFSInputStream.
2. Guard the clone call with an instanceof DFSInputStream check and handle non-DFS streams separately.

When it happens

Trigger: An open-stream clone is requested on a stream whose underlying input stream is not a DFSInputStream, so the HDFS-specific clone cannot be performed.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b2c920f0d6ac4adb. Report an issue: GitHub.