prestodb/presto · error · IllegalArgumentException

nested column [

Error message

nested column [

What it means

HiveParquetDereferencePushDown.createSubfieldColumnHandle throws IllegalArgumentException when the base (root) column of a dereferenced nested column is not present in the table scan output. Dereference push-down rewrites a nested column (e.g., a.b.c) into a subfield column handle, but it needs the parent HiveColumnHandle to exist; if baseColumnHandle is null the plan contains a nested reference with no matching base column.

Source

Thrown at presto-hive/src/main/java/com/facebook/presto/hive/rule/HiveParquetDereferencePushDown.java:90

                .getTableMetadata(session, tableHandle.getConnectorHandle())
                .getProperties());
    }

    @Override
    protected String getColumnName(ColumnHandle columnHandle)
    {
        return ((HiveColumnHandle) columnHandle).getName();
    }

    @Override
    protected ColumnHandle createSubfieldColumnHandle(
            ColumnHandle baseColumnHandle,
            Subfield subfield,
            Type subfieldDataType,
            String subfieldColumnName)
    {
        if (baseColumnHandle == null) {
            throw new IllegalArgumentException("nested column [" + subfield + "]'s base column " +
                    subfield.getRootName() + " is not present in table scan output");
        }
        HiveColumnHandle hiveBaseColumnHandle = (HiveColumnHandle) baseColumnHandle;

        Optional<HiveType> nestedColumnHiveType = hiveBaseColumnHandle
                .getHiveType()
                .findChildType(
                        subfield.getPath().stream()
                                .map(p -> ((Subfield.NestedField) p).getName())
                                .collect(Collectors.toList()));

        if (!nestedColumnHiveType.isPresent()) {
            throw new IllegalArgumentException(
                    "nested column [" + subfield + "] type is not present in Hive column type");
        }

        // Create column handle for subfield column
        return new HiveColumnHandle(

View on GitHub (pinned to 55bb57d202)

Solutions

1. Confirm the base column exists: `DESCRIBE hive.default.table` and check the root name printed in the error.
2. Fix the query to reference the actual current column names, or recreate the view with correct names.
3. If the metastore schema drifted, run MSCK REPAIR TABLE or update the metastore schema to match the data.
4. Temporarily set hive.parquet-dereference-pushdown-enabled=false to isolate/work around the optimizer issue and file a bug.
5. Refresh the plan (restart cluster or invalidate cached metadata) if a stale plan caused the mismatch.

Example fix

// before: query references old nested name after schema change
SELECT info.user.name FROM hive.default.events; -- IllegalArgumentException: base column 'info' not present
// after: use the actual column from DESCRIBE
SELECT user_info.name FROM hive.default.events;

// verify the base column resolves before issuing a dereferenced query
Row base = metadata.lookupColumn("hive", "default", "events", "info");
if (base == null) {
    throw new IllegalStateException("Base column 'info' missing from scan output; fix query/schema first");
}

// SQL-level guard: confirm base column exists in current schema before selecting subfields
boolean baseColumnExists(ConnectorSession session, SchemaTableName table, String root) {
    return metadata.listTableColumns(session, table).stream()
        .anyMatch(c -> c.getName().equalsIgnoreCase(root));
}

try {
    return planQuery(sql);
} catch (IllegalArgumentException e) {
    if (e.getMessage() != null && e.getMessage().startsWith("nested column [")) {
        throw new PlanningException("Subfield's base column not in scan output; check schema/optimizer flags", e);
    }
    throw e;
}

When it happens

Trigger: A query selects a dotted subfield of a Parquet Hive column while the base column was pruned from the scan output, or the column name doesn't resolve to a HiveColumnHandle in the current scan — e.g., after partition-column confusion, column renames, or queries where the optimizer referenced a subfield whose root isn't in the handle mapping.

Common situations: Selecting a.b.c where `a` was dropped/renamed in the metastore but the query still references the old schema; stale views or cached plans after ALTER TABLE REPLACE COLUMNS; internal optimizer interactions where column pruning removes the base column before dereference push-down runs.

Related errors

• HIVE_EXCEEDED_PARTITION_LIMIT
• HIVE_PARTITION_SCHEMA_MISMATCH
• PERMISSION_DENIED
• DRUID_QUERY_GENERATOR_FAILURE
• GENERIC_INTERNAL_ERROR

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/d563f5e2f4a2d653. Report an issue: GitHub.