{"record":{"id":"06ee10d1a7630538","repo":"influxdata/influxdb","slug":"schema-contains-non-existent-or-column","errorCode":null,"errorMessage":"schema contains non-existent or column","messagePattern":"schema contains non-existent or column","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"core/mutable_batch/src/lib.rs","lineNumber":131,"sourceCode":"    pub fn try_into_arrow(self, projection: Projection<'_>) -> Result<RecordBatch> {\n        let schema = self.schema(projection)?;\n\n        let Self {\n            column_names,\n            columns,\n            row_count: _,\n        } = self;\n\n        // Convert to Vec<Option<_>> to remove each Column avoid copying the\n        // underlying data\n        let mut columns = columns.into_iter().map(Some).collect::<Vec<_>>();\n\n        let arrays: Result<Vec<_>, Error> = schema\n            .iter()\n            .map(|(_, field)| {\n                let column_index = column_names\n                    .get(field.name())\n                    .expect(\"schema contains non-existent or column\");\n                std::mem::take(&mut columns[*column_index])\n                    .expect(\"schema contains repeated column name\")\n                    .try_into_arrow()\n                    .context(ColumnSnafu {\n                        column: field.name(),\n                    })\n            })\n            .collect();\n\n        RecordBatch::try_new(schema.into(), arrays?).context(ArrowSnafu {})\n    }\n\n    /// Returns an iterator over the columns in this batch in no particular order\n    pub fn columns(&self) -> impl ExactSizeIterator<Item = (usize, &String, &Column)> + '_ {\n        self.column_names\n            .iter()\n            .map(move |(name, idx)| (*idx, name, &self.columns[*idx]))\n    }","sourceCodeStart":113,"sourceCodeEnd":149,"githubUrl":"https://github.com/influxdata/influxdb/blob/d28e26e048401c53cbb98cf2d6ab0cf1e98048ca/core/mutable_batch/src/lib.rs#L113-L149","documentation":"MutableBatch::try_into_arrow(projection) converts the batch's columns into an Arrow RecordBatch: it builds a Schema from the projection, then for every schema field looks the column up in the batch's name-to-index map; a missing entry hits this defensive expect (the message is a typo for 'non-existent column'). schema() normally returns a proper error earlier for a projected column that does not exist, so reaching this expect means the projection and batch are inconsistent at a lower level (e.g. a schema evolved and the batch lacks the new column).","triggerScenarios":"Calling try_into_arrow(Projection::Some(&[..])) with a projection naming columns absent from the batch; converting an old-format batch against a wider table schema after a column was added, without materializing the new column.","commonSituations":"Schema evolution during writes or compaction; code building projections from a cached table schema instead of the batch itself; merging chunks with different column sets.","solutions":["Build the projection from the batch itself: use batch.columns() names or Projection::All instead of a cached table schema","Validate the projection before converting: every projected name must appear in batch.columns()","When a table gains a column, materialize it in existing batches (e.g. append a null column) before converting against the wider schema","If this panics inside released influxdb3 without custom core code, capture the batch/projection and file an issue"],"exampleFix":"// before: projection from a (newer) cached table schema\nlet rb = batch.try_into_arrow(Projection::Some(&cached_schema_cols))?;  // may name missing columns\n\n// after: project exactly what the batch has\nlet rb = batch.try_into_arrow(Projection::All)?;\n// or filter cached_schema_cols to names present in batch.columns()\n","handlingStrategy":"validation","validationCode":"// verify the projection matches the batch before converting\nlet names: HashSet<_> = batch.columns().map(|(_, n, _)| n.clone()).collect();\nfor col in projection_names {\n    assert!(names.contains(col), \"projected column {col} missing from batch\");\n}\nlet rb = batch.try_into_arrow(Projection::All)?;\n","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Derive projections from the batch (Projection::All or batch.columns()) instead of cached table schemas","When tables gain columns, materialize them (nulls) in old batches before conversion","Treat this panic as an internal invariant break - file an issue with the batch/projection if it fires in stock influxdb3"],"tags":["arrow","record-batch","mutable-batch","schema-mismatch","panic","influxdb3"],"backgroundTag":"schema-column-mismatch","analyzedSha":"d28e26e048401c53cbb98cf2d6ab0cf1e98048ca","analyzedAt":"2026-08-16T19:53:34.623Z","schemaVersion":2},"datasetVersion":"2026-08-16T23:17:17.608Z"}