apache/hadoop · error · IOException

Wrong header string: \"" + header + "\"

Error message

Wrong header string: \"" + header + "\"

What it means

S3AMultipartUploader serializes each PartHandle as a DataOutputStream payload beginning with a magic header string ("S3A-part01", S3AMultipartUploader.HEADER). parsePartHandlePayload() deserializes it during MultipartUploader.complete() and throws a plain IOException if the first UTF field is anything else. It is a format guard: the byte[] you passed as a part handle was not produced by this serializer.

Source

Thrown at hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/S3AMultipartUploader.java:320

        .toBytes();
  }

  /**
   * Parse the payload marshalled as a part handle.
   * @param data handle data
   * @return the length and etag
   * @throws IOException error reading the payload
   */
  @VisibleForTesting
  static PartHandlePayload parsePartHandlePayload(
      final byte[] data)
      throws IOException {

    try (DataInputStream input =
             new DataInputStream(new ByteArrayInputStream(data))) {
      final String header = input.readUTF();
      if (!HEADER.equals(header)) {
        throw new IOException("Wrong header string: \"" + header + "\"");
      }
      final String path = input.readUTF();
      final String uploadId = input.readUTF();
      final int partNumber = input.readInt();
      final long len = input.readLong();
      final String etag = input.readUTF();
      String checksumAlgorithm = null;
      String checksum = null;
      if (input.available() > 0) {
        checksumAlgorithm = input.readUTF();
        checksum = input.readUTF();
      }
      if (len < 0) {
        throw new IOException("Negative length");
      }
      return new PartHandlePayload(path, uploadId, partNumber, len, etag, checksumAlgorithm,
          checksum);
    }

View on GitHub (pinned to 2add963021)

Solutions

1. Use only PartHandle byte arrays returned by upload() on the same S3A MultipartUploader session, passed unchanged to complete().
2. If handles were persisted before a Hadoop upgrade, discard them, abort the uploads, and restart the upload from scratch on the current version.
3. Verify the byte[] was not re-encoded (no String round-trips with non-ISO-8859-1 charsets, no truncation) when you store it.
4. Catch IOException around complete() and fall back to abort() + full re-upload of the part.

Example fix

// before: completing with a handle of unknown provenance
partUploader.complete(destinationPath, uploadId, Arrays.asList(unknownPartHandle));

// after: validate provenance, then fall back on parse failure
try {
  partUploader.complete(destinationPath, uploadId, partsFromSameUpload);
} catch (IOException e) {
  partUploader.abort(uploadId);
  // re-upload the part with the current uploader and retry complete()
}

try {
  multipartUploader.complete(destPath, uploadHandle, partHandles);
} catch (IOException e) {
  // bad serialized part handle: abort session, re-upload parts, complete again
  multipartUploader.abort(uploadHandle);
  restartUploadFromSource();
}

When it happens

Trigger: Calling MultipartUploader.complete() with a byte[] PartHandle from a different filesystem implementation or a different MultipartUploader; handles persisted from another (older/newer) Hadoop version whose payload format changed; manually constructed or truncated handle bytes; base64/encoding round-trip that corrupted the bytes.

Common situations: Applications persisting PartHandles between jobs (e.g. writing them to a file or store and completing later across a Hadoop upgrade); mixing handles obtained from different FileSystem instances (one S3A, one wrapped/filtered or emulated); corrupted handle blobs in a queue/database.

Related errors

• Negative length
• Multipart part path mismatch: " + path
• Multipart part ID mismatch: " + uploadId
• encoded array component type {} is not a candidate primitive
• encoded array length is negative {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/d4e52ca7fe16c7a4. Report an issue: GitHub.