{"record":{"id":"d27a426a93981413","repo":"dbt-labs/dbt-core","slug":"make-encoder-for-nested-column-should-not-fail","errorCode":null,"errorMessage":"make_encoder for nested column should not fail","messagePattern":"make_encoder for nested column should not fail","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/dbt-adapter/src/record_batch.rs","lineNumber":290,"sourceCode":"impl SchemaExt for Schema {\n    fn has_dml_columns(&self, adapter_type: AdapterType) -> bool {\n        match adapter_type {\n            AdapterType::Snowflake => self\n                .fields()\n                .iter()\n                .any(|f| SNOWFLAKE_DML_COLUMNS.contains(&f.name().as_str())),\n            _ => false,\n        }\n    }\n}\n\nfn encode_array_to_strings(\n    field: &FieldRef,\n    array: &ArrayRef,\n    options: &EncoderOptions,\n) -> StringArray {\n    let mut encoder = make_encoder(field, array.as_ref(), options)\n        .expect(\"make_encoder for nested column should not fail\");\n    let mut builder = StringBuilder::with_capacity(array.len(), array.len() * 32);\n    let mut buf: Vec<u8> = Vec::with_capacity(64);\n    for row in 0..array.len() {\n        if encoder.is_null(row) {\n            builder.append_null();\n        } else {\n            buf.clear();\n            encoder.encode(row, &mut buf);\n            let s = std::str::from_utf8(&buf).expect(\"arrow_json::Encoder emits UTF-8\");\n            builder.append_value(s);\n        }\n    }\n    builder.finish()\n}\n\n/// Recursively rewrite every nested `Map` so its keys become `Utf8`.\n///\n/// arrow_json's map encoder only supports UTF-8 keys, while dbt Core stringifies any key via","sourceCodeStart":272,"sourceCodeEnd":308,"githubUrl":"https://github.com/dbt-labs/dbt-core/blob/0267ce9170576975b76b64ce856b2e5848e96617/crates/dbt-adapter/src/record_batch.rs#L272-L308","documentation":"encode_array_to_strings uses arrow_json's make_encoder to serialize a nested array (list/struct/map) into JSON strings. make_encoder returns Result and can reject unsupported DataTypes; this code asserts that for nested columns produced by jsonify_nested_columns/jsonify_map_keys the encoder is always constructible. If it returns Err, the array's DataType is not encodable — an internal invariant violation, surfaced as a panic.","triggerScenarios":"Calling encode_array_to_strings (via jsonify_nested_columns or jsonify_map_keys) on an array whose DataType arrow_json's make_encoder does not support (e.g. an exotic or newly added arrow type, or a non-nested type reaching the function by mistake).","commonSituations":"Upgrading arrow-rs introduces/renames DataTypes the encoder doesn't handle; a driver returns an unusual nested variant (e.g. large-list of views) not covered by the encoder; misrouting a plain column into the nested-encoding path.","solutions":["Print/inspect the field's DataType where it panics and confirm it is a supported nested type (Struct, List, LargeList, FixedSizeList, Map).","Upgrade or pin arrow-json to the workspace-pinned version so encoder support matches the DataTypes the adapter emits.","Extend the dispatch in jsonify_nested_columns to route unsupported DataTypes through a fallback (e.g. cast to string) instead of make_encoder.","Report the DataType to the adapter maintainers — this path is expected to always succeed."],"exampleFix":"// before\nlet mut encoder = make_encoder(field, array.as_ref(), options)\n    .expect(\"make_encoder for nested column should not fail\");\n// after\nlet mut encoder = make_encoder(field, array.as_ref(), options).map_err(|e| {\n    AdapterError::new(AdapterErrorKind::Internal, format!(\n        \"make_encoder failed for {}: {e}\", field.data_type()))\n})?;","handlingStrategy":"type-guard","validationCode":"fn is_json_encodable(dt: &DataType) -> bool {\n    matches!(dt, DataType::Struct(_) | DataType::List(_) | DataType::LargeList(_) | DataType::FixedSizeList(_, _) | DataType::Map(_, _))\n}\n// route non-nested/unsupported types away from encode_array_to_strings","typeGuard":"fn guard_nested(field: &FieldRef) -> bool {\n    matches!(field.data_type(), DataType::Struct(_) | DataType::List(_) | DataType::LargeList(_) | DataType::FixedSizeList(_, _) | DataType::Map(_, _))\n}","tryCatchPattern":null,"preventionTips":["Only call jsonify_nested_columns/jsonify_map_keys on columns matching nested DataTypes","Keep arrow-json updated with the adapter's supported DataType set","Add a fallback cast-to-string path for newly introduced arrow types","Unit-test encoding against every nested DataType your driver can return"],"tags":["arrow","arrow-json","panic","json-encoding"],"backgroundTag":"internal-invariant-violation","analyzedSha":"0267ce9170576975b76b64ce856b2e5848e96617","analyzedAt":"2026-09-07T21:53:39.732Z","contentChangedAt":"2026-09-07T21:53:39.732Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}