{"record":{"id":"d563f5e2f4a2d653","repo":"prestodb/presto","slug":"nested-column","errorCode":null,"errorMessage":"nested column [","messagePattern":"nested column \\[","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"presto-hive/src/main/java/com/facebook/presto/hive/rule/HiveParquetDereferencePushDown.java","lineNumber":90,"sourceCode":"                .getTableMetadata(session, tableHandle.getConnectorHandle())\n                .getProperties());\n    }\n\n    @Override\n    protected String getColumnName(ColumnHandle columnHandle)\n    {\n        return ((HiveColumnHandle) columnHandle).getName();\n    }\n\n    @Override\n    protected ColumnHandle createSubfieldColumnHandle(\n            ColumnHandle baseColumnHandle,\n            Subfield subfield,\n            Type subfieldDataType,\n            String subfieldColumnName)\n    {\n        if (baseColumnHandle == null) {\n            throw new IllegalArgumentException(\"nested column [\" + subfield + \"]'s base column \" +\n                    subfield.getRootName() + \" is not present in table scan output\");\n        }\n        HiveColumnHandle hiveBaseColumnHandle = (HiveColumnHandle) baseColumnHandle;\n\n        Optional<HiveType> nestedColumnHiveType = hiveBaseColumnHandle\n                .getHiveType()\n                .findChildType(\n                        subfield.getPath().stream()\n                                .map(p -> ((Subfield.NestedField) p).getName())\n                                .collect(Collectors.toList()));\n\n        if (!nestedColumnHiveType.isPresent()) {\n            throw new IllegalArgumentException(\n                    \"nested column [\" + subfield + \"] type is not present in Hive column type\");\n        }\n\n        // Create column handle for subfield column\n        return new HiveColumnHandle(","sourceCodeStart":72,"sourceCodeEnd":108,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-hive/src/main/java/com/facebook/presto/hive/rule/HiveParquetDereferencePushDown.java#L72-L108","documentation":"HiveParquetDereferencePushDown.createSubfieldColumnHandle throws IllegalArgumentException when the base (root) column of a dereferenced nested column is not present in the table scan output. Dereference push-down rewrites a nested column (e.g., a.b.c) into a subfield column handle, but it needs the parent HiveColumnHandle to exist; if baseColumnHandle is null the plan contains a nested reference with no matching base column.","triggerScenarios":"A query selects a dotted subfield of a Parquet Hive column while the base column was pruned from the scan output, or the column name doesn't resolve to a HiveColumnHandle in the current scan — e.g., after partition-column confusion, column renames, or queries where the optimizer referenced a subfield whose root isn't in the handle mapping.","commonSituations":"Selecting a.b.c where `a` was dropped/renamed in the metastore but the query still references the old schema; stale views or cached plans after ALTER TABLE REPLACE COLUMNS; internal optimizer interactions where column pruning removes the base column before dereference push-down runs.","solutions":["Confirm the base column exists: `DESCRIBE hive.default.table` and check the root name printed in the error.","Fix the query to reference the actual current column names, or recreate the view with correct names.","If the metastore schema drifted, run MSCK REPAIR TABLE or update the metastore schema to match the data.","Temporarily set hive.parquet-dereference-pushdown-enabled=false to isolate/work around the optimizer issue and file a bug.","Refresh the plan (restart cluster or invalidate cached metadata) if a stale plan caused the mismatch."],"exampleFix":"// before: query references old nested name after schema change\nSELECT info.user.name FROM hive.default.events; -- IllegalArgumentException: base column 'info' not present\n// after: use the actual column from DESCRIBE\nSELECT user_info.name FROM hive.default.events;","handlingStrategy":"validation","validationCode":"// verify the base column resolves before issuing a dereferenced query\nRow base = metadata.lookupColumn(\"hive\", \"default\", \"events\", \"info\");\nif (base == null) {\n    throw new IllegalStateException(\"Base column 'info' missing from scan output; fix query/schema first\");\n}","typeGuard":"// SQL-level guard: confirm base column exists in current schema before selecting subfields\nboolean baseColumnExists(ConnectorSession session, SchemaTableName table, String root) {\n    return metadata.listTableColumns(session, table).stream()\n        .anyMatch(c -> c.getName().equalsIgnoreCase(root));\n}","tryCatchPattern":"try {\n    return planQuery(sql);\n} catch (IllegalArgumentException e) {\n    if (e.getMessage() != null && e.getMessage().startsWith(\"nested column [\")) {\n        throw new PlanningException(\"Subfield's base column not in scan output; check schema/optimizer flags\", e);\n    }\n    throw e;\n}","preventionTips":["Keep query column names in sync with the metastore schema after ALTERs.","Rebuild dependent views after schema evolution.","If push-down misbehaves, test with hive.parquet-dereference-pushdown-enabled=false and report the repro.","Read the full parent column once to discover real nested field names."],"tags":["hive","parquet","query-planning","dereference-pushdown"],"backgroundTag":"nested-column-base-missing","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}