What causes "invalid hex digit in \u escape: %q" in vitessio/vitess?

Validate the document with encoding/json (json.Valid/Unmarshal) before passing it to the writer; the standard parser rejects malformed \\u escapes earlier with clearer context.

vitessio/vitess · error

invalid hex digit in \u escape: %q

Error message

invalid hex digit in \u escape: %q

What it means

When unescaping a JSON string body, a \u escape must be followed by exactly four hexadecimal digits. parseHex4 returned -1 because one of the four bytes after \u is not a valid hex character, so unescapeJSON rejects the string with this error. This keeps malformed or corrupted JSON strings out of the generated SQL instead of silently encoding garbage.

Source

Thrown at go/mysql/json/marshal.go:441

			dst = append(dst, '\f')
			i++
		case 'n':
			dst = append(dst, '\n')
			i++
		case 'r':
			dst = append(dst, '\r')
			i++
		case 't':
			dst = append(dst, '\t')
			i++
		case 'u':
			i++ // skip 'u'
			if i+4 > len(src) {
				return dst, errors.New("truncated \\u escape in JSON string")
			}
			r := parseHex4(src[i : i+4])
			if r < 0 {
				return dst, fmt.Errorf("invalid hex digit in \\u escape: %q", src[i:i+4])
			}
			i += 4

			// Handle UTF-16 surrogate pairs.
			if utf16.IsSurrogate(r) {
				if i+6 <= len(src) && src[i] == '\\' && src[i+1] == 'u' {
					r2 := parseHex4(src[i+2 : i+6])
					if r2 >= 0 {
						combined := utf16.DecodeRune(r, r2)
						if combined != utf8.RuneError {
							dst = utf8.AppendRune(dst, combined)
							i += 6
							continue
						}
					}
				}
				// Lone surrogate: encode as replacement character.
				dst = utf8.AppendRune(dst, utf8.RuneError)

View on GitHub (pinned to 01a25a7d17)

Solutions

Validate the document with encoding/json (json.Valid/Unmarshal) before passing it to the writer; the standard parser rejects malformed \\u escapes earlier with clearer context.
Fix the escape to use exactly four hex digits (e.g. \\u0041) or, better, re-encode the source data with json.Marshal so escapes are generated correctly.
Track down the producer emitting non-hex \\u sequences (custom escaper, template engine, log munging) and correct it.
If the data was corrupted in transit, checksum or length-validate payloads before parsing.

Example fix

// before: decimal code point in escape
s := "{\\"k\\": \\\\u12345}" // invalid
// after: proper 4-hex-digit escape
s := "{\\"k\\": \\\\u1234}"

Defensive patterns

Strategy: validation

Validate before calling

// reject malformed \\u escapes before writing
for i := 0; i+1 < len(s); i++ {
	if s[i] == '\\' && s[i+1] == 'u' {
		if i+6 > len(s) || !isHex4(s[i+2:i+6]) {
			return errors.New("invalid \\u escape")
		}
	}
}

Type guard

func hasValidUnicodeEscapes(s []byte) bool {
	for i := 0; i+1 < len(s); i++ {
		if s[i] == '\\' && s[i+1] == 'u' {
			if i+6 > len(s) {
				return false
			}
			for _, c := range s[i+2 : i+6] {
				if !((c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F')) {
					return false
				}
			}
			i += 5
		}
	}
	return true
}

Try / catch

if err := writeJSONAsSQL(input); err != nil {
	if strings.Contains(err.Error(), "invalid hex digit in \\u escape") {
		return fmt.Errorf("rejected payload: malformed unicode escape: %w", err)
	}
	return err
}

Prevention

Generate escapes with json.Marshal only
Never hand-roll \\u escaping in templates or log mungers
Decode-then-reencode suspicious payloads before passing them through
Test with emoji/CJK data to exercise unicode escapes

When it happens

Trigger: Writing a JSON string that contains an escape sequence where the four characters after \\u are not all hex digits, e.g. \\u12z4, \\u+123, \\u 1F6, or a shortened escape like \\u12 that was padded with other text. Reached via writeStringContent -> unescapeJSON when serializing a string value or an object key through the JSON-to-SQL writer.

Common situations: JSON produced by buggy custom escape routines that emit \\u followed by decimal code points instead of hex; double-encoding layers (an escaped \\u0041 turned into \\\\u0041 then corrupted); manually edited JSON fixtures with typos; corrupted binary payloads where bytes after \\u were replaced.

Related errors

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/2a01b88efdae1eff. Report an issue: GitHub.