{"record":{"id":"c3d1e1050ba40f56","repo":"risingwavelabs/risingwave","slug":"invalid-extra-partition-column-index","errorCode":null,"errorMessage":"invalid extra partition column index {}","messagePattern":"invalid extra partition column index (.+?)","errorType":"validation","errorClass":"SinkError::Iceberg","httpStatus":null,"severity":"error","filePath":"src/connector/src/sink/iceberg/writer.rs","lineNumber":702,"sourceCode":"                if writer.is_none() {\n                    *writer = Some(Box::new(\n                        writer_builder\n                            .build()\n                            .map_err(|err| SinkError::Iceberg(anyhow!(err)))?,\n                    ));\n                }\n            }\n        };\n        Ok(())\n    }\n\n    fn process_chunk(&mut self, chunk: StreamChunk) -> Result<Option<(RecordBatch, usize)>> {\n        let (mut chunk, ops) = chunk.compact_vis().into_parts();\n        match &mut self.project_idx_vec {\n            ProjectIdxVec::None => {}\n            ProjectIdxVec::Prepare(idx) => {\n                if *idx >= chunk.columns().len() {\n                    return Err(SinkError::Iceberg(anyhow!(\n                        \"invalid extra partition column index {}\",\n                        idx\n                    )));\n                }\n                let project_idx_vec = (0..*idx)\n                    .chain(*idx + 1..chunk.columns().len())\n                    .collect_vec();\n                chunk = chunk.project(&project_idx_vec);\n                self.project_idx_vec = ProjectIdxVec::Done(project_idx_vec);\n            }\n            ProjectIdxVec::Done(idx_vec) => {\n                chunk = chunk.project(idx_vec);\n            }\n        }\n        if ops.is_empty() {\n            return Ok(None);\n        }\n        let write_batch_size = chunk.estimated_heap_size();","sourceCodeStart":684,"sourceCodeEnd":720,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/connector/src/sink/iceberg/writer.rs#L684-L720","documentation":"`process_chunk` rejected a pre-computed extra-partition column projection index that is out of bounds for the current StreamChunk. When `ProjectIdxVec::Prepare(idx)` is set (the extra partition column still physically present in the chunk), `idx` must point at an existing column so it can be split out for partitioned writing. RisingWave raises `SinkError::Iceberg` with the offending index.","triggerScenarios":"`write_batch`/`write_batch_with_position` delivering a chunk with fewer columns than the stored partition-projection index expects — typically after the MV/sink projection changed while the writer still holds a stale index, or an internal bug computing `project_idx_vec`.","commonSituations":"Altering the MV or sink column list while the sink is running; sink created with extra partition columns whose index no longer matches the chunk arity; internal invariant breakage in chunk column count (e.g. degenerate single-column chunks).","solutions":["Recreate the sink so `project_idx_vec` is recomputed against the current MV projection.","Verify the MV/sink column list matches the configured extra partition columns in the sink WITH options.","If reproducible on a fresh sink, report it as an internal bug with the chunk column count and stored index."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Guard the stored index against the chunk arity before using it\nif let ProjectIdxVec::Prepare(idx) = &self.project_idx_vec {\n    anyhow::ensure!(\n        *idx < chunk.columns().len(),\n        \"extra partition column index {} out of bounds for chunk with {} columns; recreate the sink\",\n        idx, chunk.columns().len()\n    );\n}","typeGuard":"fn idx_in_bounds(idx: &usize, len: usize) -> bool { idx < &len }","tryCatchPattern":"// Wrap the write so an arity mismatch triggers sink re-creation rather than silent data loss\nmatch sink.write_batch(chunk).await {\n    Err(e) if e.to_string().contains(\"invalid extra partition column index\") => {\n        recreate_sink().await?;\n    }\n    other => other?,\n}","preventionTips":["Do not ALTER MV projections while the Iceberg sink is running","Verify extra-partition-column configuration matches the MV schema at sink creation","Add an assert/log on stored project indices when the sink is built"],"tags":["iceberg","partition","index-out-of-range","internal"],"backgroundTag":"index-out-of-range","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}