{"record":{"id":"c409bf7346ca6785","repo":"risingwavelabs/risingwave","slug":"map-is-not-supported-for-bigquery-sink","errorCode":null,"errorMessage":"MAP is not supported for BigQuery sink.","messagePattern":"MAP is not supported for BigQuery sink\\.","errorType":"validation","errorClass":"SinkError::BigQuery","httpStatus":null,"severity":"error","filePath":"src/connector/src/sink/big_query.rs","lineNumber":430,"sourceCode":"                        Self::get_string_and_check_support_from_datatype(datatype)?;\n                    elements_vec.push(format!(\"{} {}\", name, element_string));\n                }\n                Ok(format!(\"STRUCT<{}>\", elements_vec.join(\", \")))\n            }\n            DataType::List(l) => {\n                let element_string = Self::get_string_and_check_support_from_datatype(l.elem())?;\n                Ok(format!(\"ARRAY<{}>\", element_string))\n            }\n            DataType::Bytea => Ok(\"BYTES\".to_owned()),\n            DataType::Jsonb => Ok(\"JSON\".to_owned()),\n            DataType::Variant => Err(SinkError::BigQuery(anyhow::anyhow!(\n                \"VARIANT is not supported for BigQuery sink.\"\n            ))),\n            DataType::Serial => Ok(\"INT64\".to_owned()),\n            DataType::Int256 => Err(SinkError::BigQuery(anyhow::anyhow!(\n                \"INT256 is not supported for BigQuery sink.\"\n            ))),\n            DataType::Map(_) => Err(SinkError::BigQuery(anyhow::anyhow!(\n                \"MAP is not supported for BigQuery sink.\"\n            ))),\n            DataType::Vector(_) => Err(SinkError::BigQuery(anyhow::anyhow!(\n                \"VECTOR is not supported for BigQuery sink.\"\n            ))),\n        }\n    }\n\n    fn map_field(rw_field: &Field) -> Result<TableFieldSchema> {\n        let tfs = match &rw_field.data_type {\n            DataType::Boolean => TableFieldSchema::bool(&rw_field.name),\n            DataType::Int16 | DataType::Int32 | DataType::Int64 | DataType::Serial => {\n                TableFieldSchema::integer(&rw_field.name)\n            }\n            DataType::Float32 => {\n                return Err(SinkError::BigQuery(anyhow::anyhow!(\n                    \"REAL is not supported for BigQuery sink. Please convert to FLOAT64 or other supported types.\"\n                )));","sourceCodeStart":412,"sourceCodeEnd":448,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/connector/src/sink/big_query.rs#L412-L448","documentation":"The BigQuery sink's type mapping function `get_string_and_check_support_from_datatype` encountered a RisingWave `DataType::Map` column, which has no BigQuery equivalent in this sink's mapping table. BigQuery has no native MAP type, so the sink refuses to proceed rather than silently coercing data. The error is wrapped as `SinkError::BigQuery` and aborts sink creation or type validation.","triggerScenarios":"Creating or validating a BigQuery sink whose schema contains a column of RisingWave type MAP (e.g. `MAP<VARCHAR, INT>`), either at top level or encountered during schema resolution.","commonSituations":"Sinking a materialized view or table that uses map-typed columns (often produced by aggregations or JSON parsing) into BigQuery; users assume maps serialize like JSON but this sink rejects them outright.","solutions":["Change the column type to JSONB before sinking (e.g. recreate the source/materialized view with the map cast or stored as JSONB).","Restructure the query feeding the sink so map data is unnested into rows or flattened into separate scalar columns.","Cast the MAP column to VARCHAR/JSONB in the sink's query definition if the sink supports a SELECT/query-based definition."],"exampleFix":"// before: sink references a MAP column\nCREATE SINK s FROM mv WITH (...) TYPE bigquery;\n-- mv has col map_col MAP<VARCHAR,INT>\n\n// after: cast to JSONB in the materialized view feeding the sink\nCREATE MATERIALIZED VIEW mv2 AS SELECT map_to_json(map_col) AS map_col FROM mv;\nCREATE SINK s FROM mv2 WITH (...) TYPE bigquery;","handlingStrategy":"validation","validationCode":"-- Before creating the sink, verify no MAP columns exist:\nSELECT column_name, data_type\nFROM rw_catalog.rw_columns\nWHERE relation_id = ('<schema>.<mv_name>')::regclass\n  AND data_type ILIKE 'map%';","typeGuard":"fn is_bigquery_supported(dt: &DataType) -> bool {\n    !matches!(dt, DataType::Map(_) | DataType::Vector(_) | DataType::Int256 | DataType::Variant)\n}","tryCatchPattern":null,"preventionTips":["Audit sink source schemas for MAP columns before configuring BigQuery sinks.","Standardize on JSONB for semi-structured data destined for BigQuery.","Include unsupported-type checks in CI for sink definitions."],"tags":["bigquery","sink","unsupported-type","data-type"],"backgroundTag":"unsupported-operation","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}