prestodb/presto · error · PrestoException

HIVE_INVALID_ENCRYPTION_METADATA

HIVE_INVALID_ENCRYPTION_METADATA

Error message

no column found for encryption field %s

What it means

toOrcColumnIndex resolves an encryption field name (from DwrfEncryptionMetadata) to an ORC column index. It throws HIVE_INVALID_ENCRYPTION_METADATA when the column named in the encryption metadata does not exist in the Hive table's column-name-to-index mapping, meaning the encryption metadata refers to a column the table no longer (or never did) have.

Source

Thrown at presto-hive/src/main/java/com/facebook/presto/hive/DwrfEncryptionMetadata.java:167

        return toKeyMap(types, columnIndexMap);
    }

    public Map<Integer, Slice> toKeyMap(List<OrcType> types, Map<String, Integer> columnNamesToHiveIndex)
    {
        if (fieldToKeyData.containsKey(TABLE_IDENTIFIER)) {
            return ImmutableMap.of(0, Slices.wrappedBuffer(fieldToKeyData.get(TABLE_IDENTIFIER)));
        }

        return fieldToKeyData.entrySet().stream()
                .collect(toImmutableMap(entry -> toOrcColumnIndex(entry.getKey(), types, columnNamesToHiveIndex), entry -> Slices.wrappedBuffer(entry.getValue())));
    }

    private static int toOrcColumnIndex(String fieldString, List<OrcType> types, Map<String, Integer> columnNamesToHiveIndex)
    {
        ColumnEncryptionInformation.ColumnWithStructSubfield columnWithStructSubfield = ColumnEncryptionInformation.ColumnWithStructSubfield.valueOf(fieldString);

        if (!columnNamesToHiveIndex.containsKey(columnWithStructSubfield.getColumnName())) {
            throw new PrestoException(HIVE_INVALID_ENCRYPTION_METADATA, format("no column found for encryption field %s", columnWithStructSubfield.getColumnName()));
        }
        int columnRoot = columnNamesToHiveIndex.get(columnWithStructSubfield.getColumnName());
        return getOrcColumnIndexRecursive(types, types.get(0).getFieldTypeIndex(columnRoot), columnWithStructSubfield.getChildField());
    }

    private static int getOrcColumnIndexRecursive(List<OrcType> types, int typeId, Optional<ColumnEncryptionInformation.ColumnWithStructSubfield> subfield)
    {
        OrcType type = types.get(typeId);

        int columnId = typeId;
        if (subfield.isPresent()) {
            verify(type.getOrcTypeKind() == STRUCT, "subfield references are only permitted for struct types, but found %s for column %s", subfield, columnId);
            String name = subfield.get().getColumnName().toLowerCase(Locale.ENGLISH);
            Optional<ColumnEncryptionInformation.ColumnWithStructSubfield> nextSubfield = subfield.get().getChildField();

            int children = type.getFieldCount();
            for (int i = 0; i < children; ++i) {
                String fieldName = type.getFieldNames().get(i).toLowerCase(Locale.ENGLISH);

View on GitHub (pinned to 55bb57d202)

Solutions

1. Compare the column names in the table's encryption metadata (hive.encryption.metadata / ENCRYPT_COLUMNS properties) with the actual table schema and fix mismatches
2. Restore the dropped/renamed column or update the encryption metadata keys to match current column names
3. Regenerate encryption metadata by rewriting the table with correct column-level encryption properties
4. Verify column name case matches the Hive schema, since lookups are name-based

Example fix

// before: metadata references 'comments' but table has 'comment'
ENCRYPT_COLUMNS='comments'
// after: match actual Hive column name
ENCRYPT_COLUMNS='comment'

Set<String> tableColumns = tableSchema.getColumns().stream()
    .map(c -> c.getName().toLowerCase(Locale.ENGLISH))
    .collect(Collectors.toSet());
for (String field : encryptionFields) {
    String col = ColumnEncryptionInformation.ColumnWithStructSubfield.valueOf(field).getColumnName();
    if (!tableColumns.contains(col.toLowerCase(Locale.ENGLISH))) {
        throw new IllegalArgumentException("Encryption field not a table column: " + col);
    }
}

boolean columnExists(String field, Set<String> hiveColumns) {
    return hiveColumns.contains(
        ColumnEncryptionInformation.ColumnWithStructSubfield.valueOf(field).getColumnName().toLowerCase(Locale.ENGLISH));
}

try {
    readEncryptedTable(...);
} catch (PrestoException e) {
    if ("HIVE_INVALID_ENCRYPTION_METADATA".equals(e.getErrorCode().getName())) {
        // refresh/rebuild encryption metadata against current schema
    }
    throw e;
}

When it happens

Trigger: Reading a DWRF-encrypted table whose encryption metadata lists a column name absent from the current table schema — e.g. the column was dropped or renamed after encryption metadata was written, or the metadata key casing/format does not match the actual Hive column name.

Common situations: ALTER TABLE DROP/RENAME column on an encrypted table, hand-edited or corrupted serde properties, or reading a table with stale ColumnEncryptionInformation in metastore properties.

Related errors

• GENERIC_INTERNAL_ERROR
• HIVE_INVALID_ENCRYPTION_METADATA
• HIVE_INVALID_ENCRYPTION_METADATA
• HIVE_UNSUPPORTED_ENCRYPTION_OPERATION
• HIVE_PARTITION_SCHEMA_MISMATCH

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/34b36b4d2ad1c96c. Report an issue: GitHub.