What causes "Cannot truncate to a larger file size. Current size: , truncate size: ." in apache/hadoop?

Re-stat the file immediately before truncating and clamp: newLength = Math.min(newLength, fs.getFileStatus(path).getLen())

apache/hadoop · error · HadoopIllegalArgumentException

Cannot truncate to a larger file size. Current size: <oldLen

Error message

Cannot truncate to a larger file size. Current size: <oldLength>, truncate size: <newLength>.

What it means

HDFS truncate can only shrink a file. FSDirTruncateOp compares the current file size with the requested newLength after lease recovery and throws HadoopIllegalArgumentException when newLength is greater than the current size; equal lengths are a no-op that returns success. This is pure client-argument validation that fails before any block is modified.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirTruncateOp.java:133

            return new TruncateResult(false, fsd.getAuditFileInfo(iip));
          } else {
            throw new AlreadyBeingCreatedException(
                RecoverLeaseOp.TRUNCATE_FILE.getExceptionMessage(src,
                    clientName, clientMachine, src + " is being truncated."));
          }
        }
      }

      // Opening an existing file for truncate. May need lease recovery.
      fsn.recoverLeaseInternal(RecoverLeaseOp.TRUNCATE_FILE, iip, src,
          clientName, clientMachine, false);
      // Truncate length check.
      long oldLength = file.computeFileSize();
      if (oldLength == newLength) {
        return new TruncateResult(true, fsd.getAuditFileInfo(iip));
      }
      if (oldLength < newLength) {
        throw new HadoopIllegalArgumentException(
            "Cannot truncate to a larger file size. Current size: " + oldLength
                + ", truncate size: " + newLength + ".");
      }
      // Perform INodeFile truncation.
      final QuotaCounts delta = new QuotaCounts.Builder().build();
      onBlockBoundary = unprotectedTruncate(fsn, iip, newLength,
          toRemoveBlocks, mtime, delta);
      if (!onBlockBoundary) {
        // Open file for write, but don't log into edits
        long lastBlockDelta = file.computeFileSize() - newLength;
        assert lastBlockDelta > 0 : "delta is 0 only if on block bounday";
        truncateBlock = prepareFileForTruncate(fsn, iip, clientName,
            clientMachine, lastBlockDelta, null);
      }

      // update the quota: use the preferred block size for UC block
      fsd.updateCountNoQuotaCheck(iip, iip.length() - 1, delta);
    } finally {

View on GitHub (pinned to 2add963021)

Solutions

Re-stat the file immediately before truncating and clamp: newLength = Math.min(newLength, fs.getFileStatus(path).getLen())
If growing the file is the goal, use append() plus writes instead of truncate()
Derive lengths from authoritative getFileStatus() values, not cached ones

Example fix

// before
dfs.truncate(path, rememberedSize);

// after
long current = fs.getFileStatus(path).getLen();
dfs.truncate(path, Math.min(newLength, current));

Defensive patterns

Strategy: validation

Validate before calling

long current = fs.getFileStatus(path).getLen();
long target = Math.min(newLength, current);
if (target != current) {
  dfs.truncate(path, target);
}

Try / catch

try {
  dfs.truncate(path, newLength);
} catch (HadoopIllegalArgumentException e) {
  // re-stat and clamp, or switch to append() if growth was intended
  long current = fs.getFileStatus(path).getLen();
  dfs.truncate(path, Math.min(newLength, current));
}

Prevention

Never derive newLength from cached sizes; stat first
Clamp truncate targets to the range [0, current length]
Use append() to grow a file; truncate only shrinks

When it happens

Trigger: Calling DistributedFileSystem.truncate(path, newLength) with newLength greater than the file's current length: stale cached size, a race with another truncate that already shrank the file, or an offset/unit computation bug in the caller.

Common situations: Log rotation truncating to a remembered offset after the file was already cut shorter; bytes-vs-blocks math errors; retrying a truncate after a concurrent job reduced the file.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/e05ea786914a58e3. Report an issue: GitHub.