apache/hadoop · error · IllegalArgumentException

Cannot truncate to a larger file size. Current size: " + old

Error message

Cannot truncate to a larger file size. Current size: " + oldLength + ", truncate size: " + newLength + "."

What it means

RawLocalFileSystem.truncate(Path, long) throws IllegalArgumentException when newLength exceeds the file's current size: local filesystems cannot extend a file via truncate (unlike some object stores or HDFS-ordered semantics where newLength == len is the only allowed non-shrink). The message reports both current and requested sizes, which tells you exactly how far off the caller was.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java:768

        }
      }
    } catch (FileNotFoundException ignored) {
    }
    return false;
  }

  @Override
  public boolean truncate(Path f, final long newLength) throws IOException {
    FileStatus status = getFileStatus(f);
    if(status == null) {
      throw new FileNotFoundException("File " + f + " not found");
    }
    if(status.isDirectory()) {
      throw new IOException("Cannot truncate a directory (=" + f + ")");
    }
    long oldLength = status.getLen();
    if(newLength > oldLength) {
      throw new IllegalArgumentException(
          "Cannot truncate to a larger file size. Current size: " + oldLength +
          ", truncate size: " + newLength + ".");
    }
    try (FileOutputStream out = new FileOutputStream(pathToFile(f), true)) {
      try {
        out.getChannel().truncate(newLength);
      } catch(IOException e) {
        throw new FSError(e);
      }
    }
    return true;
  }
  
  /**
   * Delete the given path to a file or directory.
   * @param p the path to delete
   * @param recursive to delete sub-directories
   * @return true if the file or directory and all its contents were deleted

View on GitHub (pinned to 2add963021)

Solutions

Clamp the request: long target = Math.min(newLength, fs.getFileStatus(f).getLen()); then truncate.
If extension is the real goal, open the file in append mode and pad: fs.append(f) then write zeros.
Re-stat the file immediately before truncating to avoid stale lengths.
Fix the arithmetic: to keep N bytes, pass N; to drop K bytes, pass len - K.

Example fix

// before
fs.truncate(logPath, 50_000_000); // file is only 12_000_000 bytes -> throws

// after
long target = Math.min(50_000_000, fs.getFileStatus(logPath).getLen());
fs.truncate(logPath, target);

Defensive patterns

Strategy: validation

Validate before calling

long current = fs.getFileStatus(f).getLen();
long target = Math.min(newLength, current);
if (target != newLength) {
  LOG.warn("Truncate clamped from {} to {} (current size)", newLength, target);
}
fs.truncate(f, target);

Try / catch

try {
  fs.truncate(f, newLength);
} catch (IllegalArgumentException e) {
  if (e.getMessage() != null && e.getMessage().contains("larger file size")) {
    fs.truncate(f, fs.getFileStatus(f).getLen()); // no-op semantics
  } else {
    throw e;
  }
}

Prevention

Clamp newLength to the freshly-statted file size before calling truncate.
Remember truncate only shrinks on local FS; use append to grow.
Re-stat immediately before truncating to avoid stale cached lengths.

When it happens

Trigger: Calling fs.truncate(f, newLength) with newLength > f.length(): passing a desired total size instead of a truncation point, using a stale cached length (file shrank since stat), or computing newLength as oldLen + delta by mistake.

Common situations: Porting code that assumed truncate pads with zeros, rolling-log logic that mixes up 'target size' and 'bytes to keep', or files concurrently truncated by another process between the length check and the call.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/adabd85068b5cf27. Report an issue: GitHub.