risingwavelabs/risingwave · error · GenDataFusionPlanError

Missing Iceberg Scan

Error message

Missing Iceberg Scan

What it means

Part of `GenDataFusionPlanError`, thrown when converting an optimized RisingWave batch plan into an Apache DataFusion `LogicalPlan` for Iceberg queries. The converter expects the plan to contain an Iceberg scan node; if none is found where one is required, it returns `MissingIcebergScan`.

Solutions

1. Ensure the query targets an actual Iceberg table/source so the optimized plan contains an Iceberg scan.
2. Check the plan translation code: verify it inspects the correct plan node position for the Iceberg scan.
3. Route non-Iceberg tables to the normal batch execution engine instead of the DataFusion path.

Example fix

// before
let df_plan = try_gen_datafusion_plan(&optimized_plan)?; // plan from a non-iceberg table
// after
if !plan_is_iceberg_scan(&optimized_plan) {
    return Err(anyhow!("target must be an Iceberg table"));
}
let df_plan = try_gen_datafusion_plan(&optimized_plan)?;

// confirm the target is an Iceberg-backed source before using the DataFusion path
const src = await client.query("SELECT connector FROM rw_sources WHERE name = $1", [t]);

function isIcebergSource(row) { return row && String(row.connector).toLowerCase() === "iceberg"; }

match try_gen_datafusion_plan(&plan) {
    Err(GenDataFusionPlanError::MissingIcebergScan) => {
        // fall back to the normal batch engine
    }
    r => r?,
}

When it happens

Trigger: Calling `try_gen_datafusion_plan` on a batch optimized plan whose leaf/scan node is not an Iceberg scan (e.g. the table resolved to a regular MV or non-Iceberg source), so plan translation finds no IcebergScan to translate.

Common situations: Pointing the DataFusion-based Iceberg query path at a table that is not backed by an Iceberg scan; plan rewrites (e.g. pushed-down projections or joins) that reordered or removed the scan node expected by the translator.

Understand the failure class

Background: "not installed", "pip install", "required for": how missing-dependency errors surface across open-source libraries — this error's family across 34 libraries.

Related errors

• Generating plan error
• Unsupported Plan Node
• DataFusion error
• {0}
• adlsgen2.authority_host does not parse as a URL

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/d9af65be80336486. Report an issue: GitHub.

Appendix: source

Thrown at src/frontend/src/datafusion/execute/mod.rs:62

use crate::session::SessionImpl;
use crate::utils::DropGuard;

mod memory_ctx;
mod query_planner;

pub(crate) use memory_ctx::create_df_spillable_budget_ctx;
const DF_MANAGED_SPILL_DIR: &str = "df_batch_spill/";

#[derive(Clone)]
pub struct DfBatchQueryPlanResult {
    pub(crate) plan: Arc<LogicalPlan>,
    pub(crate) schema: RwSchema,
    pub(crate) stmt_type: StatementType,
}

#[derive(Debug, Error)]
pub enum GenDataFusionPlanError {
    #[error("Missing Iceberg Scan")]
    MissingIcebergScan,
    #[error("Unsupported Plan Node")]
    UnsupportedPlanNode,
    #[error("Generating plan error: {0}")]
    Generating(#[source] RwError),
}

pub fn try_gen_datafusion_plan(
    optimized_logical: &BatchOptimizedLogicalPlanRoot,
) -> Result<Arc<LogicalPlan>, GenDataFusionPlanError> {
    use crate::optimizer::DataFusionExecuteCheckerExt;

    let check_result = optimized_logical.plan.check_for_datafusion();
    if !check_result.have_iceberg_scan {
        return Err(GenDataFusionPlanError::MissingIcebergScan);
    }
    if !check_result.supported {
        return Err(GenDataFusionPlanError::UnsupportedPlanNode);

View on GitHub (pinned to 6469eb736d)