{"record":{"id":"21b2e759cb3cffac","repo":"risingwavelabs/risingwave","slug":"turbopuffer-document-id-column-cannot-be-null","errorCode":null,"errorMessage":"Turbopuffer document id column cannot be null","messagePattern":"Turbopuffer document id column cannot be null","errorType":"http","errorClass":"SinkError::Http","httpStatus":null,"severity":"error","filePath":"src/connector/src/sink/turbopuffer.rs","lineNumber":519,"sourceCode":"                        )));\n                    }\n                    Some(_) => {\n                        return Err(SinkError::Http(anyhow!(\n                            \"unexpected namespace_column type, expected varchar\"\n                        )));\n                    }\n                };\n                validate_namespace(namespace)?;\n                Ok(format!(\"{}/v2/namespaces/{}\", self.base_url, namespace))\n            }\n        }\n    }\n\n    // Turbopuffer document IDs are unsigned 64-bit integers, UUIDs, or strings up to 64 bytes.\n    // RisingWave UUID IDs can be represented with varchar.\n    fn id_for_row(&self, row: &impl Row) -> Result<DocumentId> {\n        let datum = row.datum_at(self.pk_index).ok_or_else(|| {\n            SinkError::Http(anyhow!(\"Turbopuffer document id column cannot be null\"))\n        })?;\n        match datum {\n            ScalarRefImpl::Int16(value) => Ok(document_id_from_i64(value as i64)),\n            ScalarRefImpl::Int32(value) => Ok(document_id_from_i64(value as i64)),\n            ScalarRefImpl::Int64(value) => Ok(document_id_from_i64(value)),\n            ScalarRefImpl::Serial(value) => Ok(document_id_from_i64(value.into_inner())),\n            ScalarRefImpl::Utf8(value) => {\n                if value.len() > 64 {\n                    return Err(SinkError::Http(anyhow!(\n                        \"Turbopuffer string document id exceeds 64 bytes\"\n                    )));\n                }\n                Ok(DocumentId::String(value.to_owned()))\n            }\n            _ => Err(SinkError::Http(anyhow!(\n                \"Turbopuffer document id column must be an integer or varchar\"\n            ))),\n        }","sourceCodeStart":501,"sourceCodeEnd":537,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/connector/src/sink/turbopuffer.rs#L501-L537","documentation":"Every Turbopuffer document needs an ID, which the sink takes from the primary-key column (`pk_index`) in `id_for_row`. If the pk datum is NULL for a row, no document ID can be built and an Http error is raised for the write.","triggerScenarios":"Writing a row whose primary-key column is NULL to a Turbopuffer sink; NULL pk values arriving from a source or MV where the pk is nullable.","commonSituations":"LEFT JOINs nulling out pk columns upstream; nullable source columns declared as pk by mistake; data corrections/backfills inserting rows with NULL keys.","solutions":["Ensure the pk column is NOT NULL upstream; drop or repair rows with NULL keys.","Pick a non-nullable column as the sink's primary key.","Add upstream filtering to skip NULL-key rows.","Re-ingest the offending rows with valid keys after fixing the source."],"exampleFix":"// before\nCREATE MATERIALIZED VIEW mv AS SELECT a, b FROM t; -- pk 'a' nullable\n\n// after\nCREATE MATERIALIZED VIEW mv AS\nSELECT * FROM t WHERE a IS NOT NULL; -- then sink pk='a'","handlingStrategy":"validation","validationCode":"// SQL: assert the pk column is non-null before sinking\nSELECT count(*) FROM mv_for_sink WHERE doc_pk IS NULL; -- should be 0","typeGuard":"function getDocId(row) {\n  const id = row.doc_pk;\n  if (id == null) throw new TypeError('document id (pk) must not be null');\n  return id;\n}","tryCatchPattern":"try {\n  await sink.write(row);\n} catch (e) {\n  if (String(e).includes('document id column cannot be null')) {\n    deadLetter.push({ row, reason: 'null-document-id' });\n  } else { throw e; }\n}","preventionTips":["Choose NOT NULL columns as the sink primary key","Filter out NULL-key rows upstream (WHERE pk IS NOT NULL)"],"tags":["sink","turbopuffer","null-value","primary-key","runtime"],"backgroundTag":"null-argument","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}