prestodb/presto · error · ParquetCryptoRuntimeException

ColumnMetaData not set in Encryption with Footer key

Error message

ColumnMetaData not set in Encryption with Footer key

What it means

When a column is encrypted with the footer key, the thrift EncryptionAlgorithm's metadata must carry the ColumnMetaData (the column's schema/type info) since it can't be read from the encrypted footer. This error is thrown when that ColumnMetaData field is missing, so the column path and type cannot be reconstructed.

Source

Thrown at presto-parquet/src/main/java/com/facebook/presto/parquet/cache/MetadataReader.java:227

                    ColumnMetaData metaData = columnChunk.meta_data;
                    ColumnCryptoMetaData cryptoMetaData = columnChunk.getCrypto_metadata();
                    ColumnPath columnPath = null;
                    boolean encryptedMetadata = false;

                    if (null == cryptoMetaData) { // Plaintext column
                        columnPath = getPath(metaData);
                        if (fileDecryptor.isPresent() && !fileDecryptor.get().plaintextFile()) {
                            // mark this column as plaintext in encrypted file decryptor
                            fileDecryptor.get().setColumnCryptoMetadata(columnPath, false, false, (byte[]) null, columnOrdinal);
                        }
                    }
                    else {  // Encrypted column
                        if (cryptoMetaData.isSetENCRYPTION_WITH_FOOTER_KEY()) { // Column encrypted with footer key
                            if (!encryptedFooter) {
                                throw new ParquetCryptoRuntimeException("Column encrypted with footer key in file with plaintext footer");
                            }
                            if (null == metaData) {
                                throw new ParquetCryptoRuntimeException("ColumnMetaData not set in Encryption with Footer key");
                            }
                            if (!fileDecryptor.isPresent()) {
                                throw new ParquetCryptoRuntimeException("Column encrypted with footer key: No keys available");
                            }
                            columnPath = getPath(metaData);
                            fileDecryptor.get().setColumnCryptoMetadata(columnPath, true, true, (byte[]) null, columnOrdinal);
                        }
                        else { // Column encrypted with column key
                            try {
                                // TODO: We decrypted data before filter projection. This could send unnecessary traffic to KMS. This so far not seen a problem in production.
                                // In parquet-mr, it uses lazy decryption but that required to change ColumnChunkMetadata. We will improve it later.
                                EncryptionWithColumnKey columnKeyStruct = cryptoMetaData.getENCRYPTION_WITH_COLUMN_KEY();
                                List<String> pathList = columnKeyStruct.getPath_in_schema();
                                byte[] columnKeyMetadata = columnKeyStruct.getKey_metadata();
                                columnPath = ColumnPath.get(pathList.toArray(new String[pathList.size()]));
                                metaData = decryptMetadata(rowGroup, columnKeyMetadata, columnChunk, fileDecryptor.get(), columnOrdinal, columnPath);
                            }
                            catch (KeyAccessDeniedException e) {

View on GitHub (pinned to 55bb57d202)

Solutions

  1. Rewrite the file with a spec-compliant encryption writer that populates ColumnMetaData for footer-key-encrypted columns
  2. Validate the file with parquet-tools / parquet-mr metadata reader to confirm corruption
  3. If the source is a pipeline, check for truncation or bad conversion steps upstream
  4. Upgrade the writer library to a version with the encryption-spec bug fixed
Defensive patterns

Strategy: try-catch

Try / catch

try {
    readParquetMetadata(dataSource);
} catch (ParquetCryptoRuntimeException e) {
    if (e.getMessage().contains("ColumnMetaData not set")) {
        // file written by a nonconformant writer; mark unreadable and report
        reportUnreadableFile(path, e);
    } else throw e;
}

Prevention

When it happens

Trigger: During convertToParquetMetadata, a column cryptoMetaData with ENCRYPTION_WITH_FOOTER_KEY is encountered and the associated metaData object is null — i.e. the writer omitted ColumnMetaData from the encrypted column metadata.

Common situations: Files produced by buggy or nonconformant encryption writers; truncated/corrupted Parquet files where thrift fields failed to deserialize; hand-crafted or converted files missing required encrypted-metadata structures.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/0eee507f52504968. Report an issue: GitHub.