prestodb/presto · error · IOException

%s header is not set on an encrypted object: %s

Error message

%s header is not set on an encrypted object: %s

What it means

getObjectSize throws IOException when an object carries the SSE customer-key user-metadata (server-side encryption header) but lacks the Unencrypted-Content-Length header. With such encryption the stored content length is not the plaintext size, so Presto cannot determine the file size without that recorded header.

Source

Thrown at presto-hive/src/main/java/com/facebook/presto/hive/s3/PrestoS3FileSystem.java:414

        }
        catch (IllegalArgumentException e) {
            log.debug(e, "Failed to parse contentType [%s], assuming not a directory", objectMetadata.getContentType());
            return false;
        }

        return mediaType.is(X_DIRECTORY_MEDIA_TYPE) ||
                (mediaType.is(OCTET_STREAM_MEDIA_TYPE)
                        && metadata.isKeyNeedsPathSeparator()
                        && objectMetadata.getContentLength() == 0);
    }

    private static long getObjectSize(Path path, ObjectMetadata metadata)
            throws IOException
    {
        Map<String, String> userMetadata = metadata.getUserMetadata();
        String length = userMetadata.get(UNENCRYPTED_CONTENT_LENGTH);
        if (userMetadata.containsKey(SERVER_SIDE_ENCRYPTION) && length == null) {
            throw new IOException(format("%s header is not set on an encrypted object: %s", UNENCRYPTED_CONTENT_LENGTH, path));
        }
        return (length != null) ? Long.parseLong(length) : metadata.getContentLength();
    }

    @Override
    public FSDataInputStream open(Path path, int bufferSize)
    {
        return new FSDataInputStream(
                new BufferedFSInputStream(
                        new PrestoS3InputStream(s3, getBucketName(uri), path, maxAttempts, maxBackoffTime, maxRetryTime),
                        bufferSize));
    }

    @Override
    public FSDataOutputStream create(Path path, FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, Progressable progress)
            throws IOException
    {
        if ((!overwrite) && exists(path)) {

View on GitHub (pinned to 55bb57d202)

Solutions

1. Re-upload the object with the UNENCRYPTED_CONTENT_LENGTH user metadata set to the plaintext size.
2. Remove SSE-C encryption or re-encrypt server-side (SSE-S3/KMS) so content length is directly readable.
3. Copy the object with metadata preservation enabled (aws s3 cp --metadata).
4. Verify with aws s3api head-object that the x-amz-meta-unencrypted-content-length header is present.

Example fix

// before (uploaded encrypted object without length header)
PutObjectRequest req = new PutObjectRequest(bucket, key, file);
req.setSseAwsKeyManagementId(...); // or SSE-C without user metadata
// after
ObjectMetadata md = new ObjectMetadata();
md.setUserMetadata(Collections.singletonMap("X-Amz-Meta-Unencrypted-Content-Length", String.valueOf(file.length())));
req.setMetadata(md);

ObjectMetadata md = s3.getObjectMetadata(bucket, key);
boolean sseC = md.getUserMetadata().containsKey("amazon-server-side-encryption") ||
              md.getSSEAlgorithm() == null && md.getUserMetadata().keySet().stream().anyMatch(k -> k.toLowerCase().endsWith("server-side-encryption"));
if (sseC && md.getUserMetadata().get("X-Amz-Meta-Unencrypted-Content-Length") == null) {
    throw new IllegalStateException("encrypted object missing length header: " + key);
}

boolean isReadableEncryptedObject(ObjectMetadata md) {
    Map<String,String> um = md.getUserMetadata();
    return !um.containsKey("X-Amz-Meta-Server-Side-Encryption") || um.get("X-Amz-Meta-Unencrypted-Content-Length") != null;
}

try {
    FileStatus st = fs.getFileStatus(path);
} catch (IOException e) {
    if (e.getMessage().contains("header is not set")) {
        // re-upload with unencrypted-content-length metadata or without SSE-C
    }
    throw e;
}

When it happens

Trigger: Reading a file that was uploaded with SSE-C client-side encryption headers by another tool that did not set the x-amz-meta-unencrypted-content-length user metadata, then calling getFileStatus/open on it.

Common situations: Files migrated from older Hadoop-S3 clients using s3a SSE-C settings; external uploads encrypted with SSE-C without emulating the Hadoop metadata convention; bucket copy operations that drop user metadata.

Related errors

• GENERIC_INTERNAL_ERROR
• Invalid encryption materials provider class:
• Unable to load or create S3 encryption materials provider: $
• ColumnMetaData not set in Encryption with Footer key
• ACCUMULO_TABLE_DNE

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/a9629a24a01a5379. Report an issue: GitHub.