meta client is required for iceberg pk-index compaction reso

Error message

meta client is required for iceberg pk-index compaction resolver

What it means

The iceberg pk-index compaction resolver executor requires a meta client to coordinate compaction via the meta service. During stream executor construction (`new_boxed_executor`), the code reads `params.env.meta_client()` and fails fast with this anyhow error when the environment has no meta client (e.g. embedded/test mode or a misconfigured compute node). This is a startup-time guard: the executor cannot function without cluster coordination.

Source

Thrown at src/stream/src/from_proto/iceberg_with_pk_index/compaction_resolver.rs:78

        let pk_data_types = node
            .pk_columns
            .iter()
            .map(|column| {
                column
                    .column_desc
                    .as_ref()
                    .map(ColumnDesc::from)
                    .map(|column| column.data_type)
                    .ok_or_else(|| anyhow!("compaction resolver PK column missing column_desc"))
            })
            .collect::<Result<Vec<_>, _>>()?;

        let barrier_receiver = params
            .local_barrier_manager
            .subscribe_barrier(params.actor_context.id);
        let local_barrier_manager = params.local_barrier_manager.clone();
        let meta_client = params.env.meta_client().ok_or_else(|| {
            anyhow!("meta client is required for iceberg pk-index compaction resolver")
        })?;
        let exec = CompactionResolverExecutor::new(
            params.actor_context,
            sink_id,
            iceberg_config,
            pk_indices,
            pk_data_types,
            params.config.developer.chunk_size,
            local_barrier_manager,
            barrier_receiver,
            meta_client,
        );
        Ok((params.info, exec).into())
    }
}

View on GitHub (pinned to 6469eb736d)

Solutions

Run the compute node against a real RisingWave cluster so `env.meta_client()` is populated (proper risedev profile, e.g. `./risedev d`).
If in tests, provide a mock/in-memory meta client when building the stream environment instead of omitting it.
Verify the executor env construction path passes a meta client for sinks requiring cluster coordination.
If the sink is not needed, remove the Iceberg pk-index sink from the streaming graph.

Example fix

// before
let env = StreamEnvironment::new(config, None); // no meta client
// after
let env = StreamEnvironment::new(config, Some(Arc::new(MetaClient::new(...))));

Defensive patterns

Strategy: validation

Validate before calling

// before building the executor
assert!(params.env.meta_client().is_some(), "this graph requires a meta client environment");

Type guard

fn has_meta_client(env: &StreamEnvironment) -> bool { env.meta_client().is_some() }

Try / catch

match env.meta_client() { Some(c) => build_with(c), None => return Err(anyhow!("meta client is required for iceberg pk-index compaction resolver")) }

Prevention

Always run Iceberg pk sinks inside a real cluster profile (`./risedev d`), not bare test envs.
In tests, use environment builders that inject a mock meta client.
Add a startup check that logs when a graph contains pk-index sinks but no meta client.

When it happens

Trigger: Building an `IcebergWithPkIndexCompactionResolver` proto node into an executor when `params.env.meta_client()` returns `None` — i.e. the compute node's environment was constructed without a meta client, typically in unit tests, embedded runtimes, or when the stream env was built with meta client disabled.

Common situations: Running a stream graph containing an Iceberg sink with primary key in an embedded/test harness without meta; launching risectl/local simulation where `env.meta_client()` is None; misconfigured compute-node startup that skips meta client construction.

Understand the failure class

Background: "not installed", "pip install", "required for": how missing-dependency errors surface across open-source libraries — this error's family across 34 libraries.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/451e3732f8efea1b. Report an issue: GitHub.