apache/hadoop · error · RuntimeException

part size mismatched: %d != %d

Error message

part size mismatched: %d != %d

What it means

FileStore.checkPartFile() validates every staged part during completeUpload: if the size recorded in the caller's Part object differs from the actual on-disk part file length it throws 'part size mismatched'. The Part metadata you passed in and the bytes staged under __STAGING__ disagree, so the object would be assembled from unverified data.

Source

Thrown at hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/object/FileStore.java:473

    return getFileChecksum(keyPath);
  }

  private byte[] getFileChecksum(Path keyPath) {
    return getFileMD5(keyPath);
  }

  private static byte[] getFileMD5(Path keyPath) {
    try {
      return DigestUtils.md5(Files.readAllBytes(keyPath));
    } catch (IOException e) {
      throw new RuntimeException(e);
    }
  }

  private static void checkPartFile(Part part, File partFile) throws IOException {
    if (part.size() != partFile.length()) {
      throw new RuntimeException(String.format("part size mismatched: %d != %d",
          part.size(), partFile.length()));
    }

    try (FileInputStream inputStream = new FileInputStream(partFile)) {
      String md5Hex = DigestUtils.md5Hex(inputStream);
      if (!Objects.equals(part.eTag(), md5Hex)) {
        throw new RuntimeException(String.format("part etag mismatched: %s != %s",
            part.eTag(), md5Hex));
      }
    }
  }

  private List<Integer> listPartNums(File uploadDir) {
    try (Stream<Path> stream = Files.list(uploadDir.toPath())) {
      return stream
          .map(f -> Integer.valueOf(f.toFile().getName()))
          .collect(Collectors.toList());
    } catch (IOException e) {

View on GitHub (pinned to 2add963021)

Solutions

1. Never construct Part objects by hand — use only the Part instances returned by uploadPart()/uploadPartCopy(), whose size/eTag reflect what was staged
2. For uploadPartCopy, verify range math: staged size is copySourceRangeEnd - copySourceRangeStart + 1 (inclusive end), clamped to the source object length via head(srcKey)
3. If any part upload was retried after a partial write, abort the upload and redo all parts under a new uploadId
4. Ensure nothing truncates or appends to files inside __STAGING__ between upload and complete

Example fix

// before: off-by-one copy range (exclusive end assumed)
long end = start + len; // declared size ends up len+1
Part p = storage.uploadPartCopy(srcKey, dstKey, uploadId, n, start, end); // part size mismatched
// after: derive the inclusive end from the real source length
ObjectInfo src = storage.head(srcKey);
long last = Math.min(start + len - 1, src.size() - 1);
Part p = storage.uploadPartCopy(srcKey, dstKey, uploadId, n, start, last);

ObjectInfo src = storage.head(srcKey);
long last = Math.min(copySourceRangeEnd, src.size() - 1);
if (last < copySourceRangeStart) {
  throw new IOException("empty or inverted copy range: [" + copySourceRangeStart + "," + last + "]");
}
long expected = last - copySourceRangeStart + 1;
Part p = storage.uploadPartCopy(srcKey, dstKey, uploadId, partNum, copySourceRangeStart, last);
if (p.size() != expected) {
  throw new IOException("part " + partNum + " size " + p.size() + " != expected " + expected);
}

When it happens

Trigger: completeUpload where a Part.size() does not match its staged file: uploadPartCopy range arithmetic producing a declared size of (end - start + 1) while a different number of bytes was staged, a truncated/extended part file from a failed or retried upload, or hand-constructed Part objects with wrong sizes.

Common situations: Off-by-one errors in copySourceRangeEnd (inclusive end vs exclusive length); retrying uploadPart after a partial failure where the part file already contains some bytes (append path); mixing Part metadata from a different upload attempt with the current staging directory.

Related errors

• parts length mismatched: %d != %d
• part num mismatched: %d != %d
• part etag mismatched: %s != %s
• Failed to create MultipartUpload with key: %s
• cannot locate the upload id: %s

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/e95be3b33c78f0fd. Report an issue: GitHub.