apache/hadoop · error · IOException

Need %d bytes, but only %d bytes available

Error message

Need %d bytes, but only %d bytes available

What it means

Error "Need %d bytes, but only %d bytes available" thrown in apache/hadoop.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/LocalReplicaInPipeline.java:212

  @Override
  public ChunkChecksum getLastChecksumAndDataLen() {
    lock.lock();
    try {
      return new ChunkChecksum(getBytesOnDisk(), lastChecksum);
    } finally {
      lock.unlock();
    }
  }

  @Override
  public void waitForMinLength(long minLength, long time, TimeUnit unit)
      throws IOException {
    long nanos = unit.toNanos(time);
    lock.lock();
    try {
      while (bytesOnDisk < minLength) {
        if (nanos <= 0L) {
          throw new IOException(
              String.format("Need %d bytes, but only %d bytes available",
                  minLength, bytesOnDisk));
        }
        nanos = bytesOnDiskChange.awaitNanos(nanos);
      }
    } catch (InterruptedException e) {
      throw new IOException(e);
    } finally {
      lock.unlock();
    }
  }

  @Override // ReplicaInPipeline
  public void setWriter(Thread writer) {
    this.writer.set(writer);
  }

  @Override

View on GitHub (pinned to 2add963021)

Solutions

1. Free space on the volume or add capacity; the required bytes exceed the available space for the RBW replica.
2. Adjust dfs.datanode.du.reserved if reserved space is consuming capacity needed for writes.

When it happens

Trigger: Writing/appending to a replica-in-pipeline needs more bytes than remain available on the volume after accounting for reserved space.

Common situations: The volume lacks the free space needed for the replica. Add capacity, rebalance, or lower replication to free space.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/d4b7593c45943e4c. Report an issue: GitHub.