{"record":{"id":"581fdd8ba86d2443","repo":"prestodb/presto","slug":"clickhouse-query-generator-failure","errorCode":"CLICKHOUSE_QUERY_GENERATOR_FAILURE","errorMessage":"Expected to find a clickhouse table scan node","messagePattern":"Expected to find a clickhouse table scan node","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"error","filePath":"presto-clickhouse/src/main/java/com/facebook/presto/plugin/clickhouse/optimization/ClickHouseComputePushdown.java","lineNumber":170,"sourceCode":"\n        public Visitor(Map<PlanNodeId, TableScanNode> tableScanNodes, ConnectorSession session, PlanNodeIdAllocator idAllocator)\n        {\n            this.session = requireNonNull(session, \"session is null\");\n            this.idAllocator = requireNonNull(idAllocator, \"idAllocator is null\");\n            this.tableScanNodes = tableScanNodes;\n            tableScanNodes.forEach((key, value) -> getClickHouseTableHandle(value).get().getTableName());\n        }\n\n        private Optional<PlanNode> tryCreatingNewScanNode(PlanNode plan)\n        {\n            Optional<ClickHouseQueryGenerator.ClickHouseQueryGeneratorResult> clickhouseSQL = clickhouseQueryGenerator.generate(plan, session);\n            if (!clickhouseSQL.isPresent()) {\n                return Optional.empty();\n            }\n            ClickHouseQueryGeneratorContext context = clickhouseSQL.get().getContext();\n            final PlanNodeId tableScanNodeId = context.getTableScanNodeId().orElseThrow(() -> new PrestoException(CLICKHOUSE_QUERY_GENERATOR_FAILURE, \"Expected to find a clickhouse table scan node id\"));\n            if (!tableScanNodes.containsKey(tableScanNodeId)) {\n                throw new PrestoException(CLICKHOUSE_QUERY_GENERATOR_FAILURE, \"Expected to find a clickhouse table scan node\");\n            }\n\n            final TableScanNode tableScanNode = tableScanNodes.get(tableScanNodeId);\n            ClickHouseTableHandle clickHouseTableHandle = getClickHouseTableHandle(tableScanNode).orElseThrow(() -> new PrestoException(CLICKHOUSE_QUERY_GENERATOR_FAILURE, \"Expected to find a clickhouse table handle\"));\n            TableHandle oldTableHandle = tableScanNode.getTable();\n            Map<VariableReferenceExpression, ClickHouseColumnHandle> assignments = context.getAssignments();\n\n            ClickHouseTableHandle oldConnectorTable = (ClickHouseTableHandle) oldTableHandle.getConnectorHandle();\n            ClickHouseTableLayoutHandle oldTableLayoutHandle = (ClickHouseTableLayoutHandle) oldTableHandle.getLayout().get();\n            ClickHouseTableLayoutHandle newTableLayoutHandle = new ClickHouseTableLayoutHandle(\n                    oldConnectorTable,\n                    oldTableLayoutHandle.getTupleDomain(),\n                    Optional.empty(), Optional.empty(), Optional.of(clickhouseSQL.get().getGeneratedClickhouseSQL()));\n\n            TableHandle newTableHandle = new TableHandle(\n                    oldTableHandle.getConnectorId(),\n                    new ClickHouseTableHandle(clickHouseTableHandle.getConnectorId(),\n                            new SchemaTableName(clickHouseTableHandle.getSchemaName(), clickHouseTableHandle.getTableName()),","sourceCodeStart":152,"sourceCodeEnd":188,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-clickhouse/src/main/java/com/facebook/presto/plugin/clickhouse/optimization/ClickHouseComputePushdown.java#L152-L188","documentation":"ClickHouseComputePushdown.tryCreatingNewScanNode() builds the pushed-down plan by replacing the original ClickHouse table scan node. If the generator context lacks a table scan node id, or the resolved id is not among the collected tableScanNodes, it throws CLICKHOUSE_QUERY_GENERATOR_FAILURE — meaning the pushdown optimizer could not locate the scan node to rewrite. This indicates an unexpected plan shape and is a connector-internal failure.","triggerScenarios":"A query eligible for ClickHouse compute pushdown whose plan contains the ClickHouse table handle but where the plan-node id recorded in the query-generation context does not match any TableScanNode in the plan (unexpected plan transformations between generation and rewrite).","commonSituations":"Plans altered by other rules/optimizers (e.g. additional exchanges or node rewrites) breaking the recorded id; queries over views or complex wrappers around the scan; connector bug on certain plan shapes.","solutions":["Disable ClickHouse compute pushdown (catalog property for pushdown/optimization) to fall back to regular execution","Simplify the query (remove intermediate views/wrappers) so the scan node survives optimization","Retry with a rewritten query form","File a connector bug with the query and EXPLAIN plan"],"exampleFix":"// before (catalog properties)\nconnector.name=clickhouse\n# pushdown enabled\n// after\nconnector.name=clickhouse\npresto-clickhouse.compute-pushdown-enabled=false","handlingStrategy":"try-catch","validationCode":"// Sanity-check plan before enabling compute pushdown\nPlanNodeId scanId = context.getTableScanNodeId().orElse(null);\nif (scanId == null || !tableScanNodes.containsKey(scanId)) {\n    log.warn(\"ClickHouse pushdown skipped: table scan node not found in plan\");\n    return Optional.empty();\n}","typeGuard":null,"tryCatchPattern":"try {\n    plan = createPushedDownPlan(...);\n} catch (PrestoException e) {\n    if (e.getErrorCode().getCode() == CLICKHOUSE_QUERY_GENERATOR_FAILURE.toErrorCode().getCode()\n            && e.getMessage().contains(\"Expected to find a clickhouse table scan node\")) {\n        return originalPlan; // fall back to unpushed plan\n    }\n    throw e;\n}","preventionTips":["Keep compute pushdown disabled in catalogs with complex plan transformations","Test pushdown queries with EXPLAIN before production","Simplify views/wrappers around ClickHouse scans","File connector bugs with plans when this recurs"],"tags":["optimizer","pushdown","plan-rewriting","internal-error"],"backgroundTag":"pushdown-plan-rewrite-failure","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}