toon-format/toon · error · TypeError

Cannot encode ${context} containing an unpaired surrogate U+

Error message

Cannot encode ${context} containing an unpaired surrogate U+${code.toString(16).toUpperCase()} at index ${index}

What it means

A TypeError (not ToonDecodeError) thrown by assertNoLoneSurrogate during value normalization. JavaScript strings can contain lone UTF-16 surrogate code units (U+D800–U+DFFF) that are not part of a valid surrogate pair; such strings cannot be encoded as valid UTF-8 TOON output, so encoding fails fast with the code unit and index. It is raised while encoding string values via normalizeValue.

Source

Thrown at packages/toon/src/encode/normalize.ts:113

function assertNoLoneSurrogate(value: string, context: string): void {
  if (!SURROGATE_PATTERN.test(value)) {
    return
  }

  for (let index = 0; index < value.length; index++) {
    const code = value.charCodeAt(index)
    if (code < 0xD800 || code > 0xDFFF) {
      continue
    }

    const isHighSurrogate = code <= 0xDBFF
    const next = value.charCodeAt(index + 1)
    if (isHighSurrogate && next >= 0xDC00 && next <= 0xDFFF) {
      index++
      continue
    }

    throw new TypeError(
      `Cannot encode ${context} containing an unpaired surrogate U+${code.toString(16).toUpperCase()} at index ${index}`,
    )
  }
}

// #endregion

// #region Type guards

export function isJsonPrimitive(value: unknown): value is JsonPrimitive {
  return (
    value === null
    || typeof value === 'string'
    || typeof value === 'number'
    || typeof value === 'boolean'
  )
}

View on GitHub (pinned to 604eac266e)

Solutions

1. Fix the data source so the string contains only well-formed Unicode (use String.prototype.isWellFormed() / toWellFormed() to sanitize)
2. Avoid cutting surrogate pairs: iterate with for...of or Intl.Segmenter instead of index-based slice
3. Replace or strip the offending code unit at the reported index before encoding

Example fix

// before
const label = raw.slice(0, 5) // may split a surrogate pair
encoder.encode({ label })
// after
const label = raw.toWellFormed().slice(0, 5)
encoder.encode({ label })

for (const [key, value] of Object.entries(data)) {
  if (typeof value === 'string' && !value.isWellFormed()) throw new Error(`lone surrogate in ${key}`)
}

function isWellFormedString(v: unknown): v is string { return typeof v === 'string' && v.isWellFormed() }

try { encode(data) } catch (e) { if (e instanceof TypeError && e.message.includes('unpaired surrogate')) { const m = e.message.match(/U\+([0-9A-F]+) at index (\d+)/); console.error('bad surrogate', m?.[1], 'at index', m?.[2]); data = sanitize(data) } }

When it happens

Trigger: Calling TOON encode with a string value (or an object/array containing one) that includes an unpaired high or low surrogate — typically from slicing a string in the middle of a surrogate pair, decoding invalid bytes, or a single half of an emoji being isolated.

Common situations: String.slice/substring/substr cutting through an emoji or non-BMP character; binary data decoded with utf8 replacement gone wrong; data received from databases or APIs containing corrupted Unicode; regex operations on code-unit indices.

Related errors

• Invalid escape sequence: \u${hex} is a lone surrogate. Suppl
• Top-level document must start with a key-value or array-head
• Indentation depth jump: expected depth ${parentDepth + 1}, b
• Unexpected bare token line outside root primitive position
• Unexpected content after the document root

AI-assisted analysis of toon-format/toon@604eac266e (2026-08-31). Data as JSON: /api/errors/19a1aa9113c1bb40. Report an issue: GitHub.