{"record":{"id":"19a1aa9113c1bb40","repo":"toon-format/toon","slug":"cannot-encode-context-containing-an-unpaired-su","errorCode":null,"errorMessage":"Cannot encode ${context} containing an unpaired surrogate U+${code.toString(16).toUpperCase()} at index ${index}","messagePattern":"Cannot encode (.+?) containing an unpaired surrogate U\\+(.+?) at index (.+?)","errorType":"exception","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"packages/toon/src/encode/normalize.ts","lineNumber":113,"sourceCode":"function assertNoLoneSurrogate(value: string, context: string): void {\n  if (!SURROGATE_PATTERN.test(value)) {\n    return\n  }\n\n  for (let index = 0; index < value.length; index++) {\n    const code = value.charCodeAt(index)\n    if (code < 0xD800 || code > 0xDFFF) {\n      continue\n    }\n\n    const isHighSurrogate = code <= 0xDBFF\n    const next = value.charCodeAt(index + 1)\n    if (isHighSurrogate && next >= 0xDC00 && next <= 0xDFFF) {\n      index++\n      continue\n    }\n\n    throw new TypeError(\n      `Cannot encode ${context} containing an unpaired surrogate U+${code.toString(16).toUpperCase()} at index ${index}`,\n    )\n  }\n}\n\n// #endregion\n\n// #region Type guards\n\nexport function isJsonPrimitive(value: unknown): value is JsonPrimitive {\n  return (\n    value === null\n    || typeof value === 'string'\n    || typeof value === 'number'\n    || typeof value === 'boolean'\n  )\n}\n","sourceCodeStart":95,"sourceCodeEnd":131,"githubUrl":"https://github.com/toon-format/toon/blob/604eac266e35166bed6f5b9e3bd586e9c60a9330/packages/toon/src/encode/normalize.ts#L95-L131","documentation":"A TypeError (not ToonDecodeError) thrown by assertNoLoneSurrogate during value normalization. JavaScript strings can contain lone UTF-16 surrogate code units (U+D800–U+DFFF) that are not part of a valid surrogate pair; such strings cannot be encoded as valid UTF-8 TOON output, so encoding fails fast with the code unit and index. It is raised while encoding string values via normalizeValue.","triggerScenarios":"Calling TOON encode with a string value (or an object/array containing one) that includes an unpaired high or low surrogate — typically from slicing a string in the middle of a surrogate pair, decoding invalid bytes, or a single half of an emoji being isolated.","commonSituations":"String.slice/substring/substr cutting through an emoji or non-BMP character; binary data decoded with utf8 replacement gone wrong; data received from databases or APIs containing corrupted Unicode; regex operations on code-unit indices.","solutions":["Fix the data source so the string contains only well-formed Unicode (use String.prototype.isWellFormed() / toWellFormed() to sanitize)","Avoid cutting surrogate pairs: iterate with for...of or Intl.Segmenter instead of index-based slice","Replace or strip the offending code unit at the reported index before encoding"],"exampleFix":"// before\nconst label = raw.slice(0, 5) // may split a surrogate pair\nencoder.encode({ label })\n// after\nconst label = raw.toWellFormed().slice(0, 5)\nencoder.encode({ label })","handlingStrategy":"validation","validationCode":"for (const [key, value] of Object.entries(data)) {\n  if (typeof value === 'string' && !value.isWellFormed()) throw new Error(`lone surrogate in ${key}`)\n}","typeGuard":"function isWellFormedString(v: unknown): v is string { return typeof v === 'string' && v.isWellFormed() }","tryCatchPattern":"try { encode(data) } catch (e) { if (e instanceof TypeError && e.message.includes('unpaired surrogate')) { const m = e.message.match(/U\\+([0-9A-F]+) at index (\\d+)/); console.error('bad surrogate', m?.[1], 'at index', m?.[2]); data = sanitize(data) } }","preventionTips":["Call String.prototype.toWellFormed() on external string input before encoding","Slice strings with code-point-aware methods (for...of, [...str], Intl.Segmenter), not raw charCodeAt indices","Beware of splitting emojis/CJK characters with slice/substring","Validate Unicode well-formedness at API boundaries"],"tags":["unicode","surrogate","encoding","toon"],"backgroundTag":"lone-surrogate","analyzedSha":"604eac266e35166bed6f5b9e3bd586e9c60a9330","analyzedAt":"2026-08-31T11:09:25.043Z","schemaVersion":2},"datasetVersion":"2026-09-01T08:17:40.651Z"}