What causes "Invalid character code point %d in string" in antonmedv/fx?

Transcode the input to valid UTF-8 before parsing (e.g. golang.org/x/text/encoding transform from the actual source encoding)

antonmedv/fx · error

Invalid character code point %d in string

Error message

Invalid character code point %d in string

What it means

scanString read a byte whose value, interpreted as a rune, exceeds unicode.MaxRune (0x10FFFF) — i.e. invalid UTF-8 / an out-of-range code point inside a string literal. The library rejects it because valid JSON strings must contain valid Unicode.

Source

Thrown at internal/jsonx/json.go:241

						s += string(p.char)
					}
					_, err := strconv.ParseInt(s, 16, 32)
					if err != nil {
						panic(fmt.Sprintf("Invalid Unicode escape sequence '\\u%s'", s))
					}
				case '"', '\\', '/', 'b', 'f', 'n', 'r', 't':
				default:
					panic(fmt.Sprintf("Invalid escape sequence '\\%c'", p.char))
				}
			}
		} else if p.char == '\\' {
			escaped = true
		} else if p.char == '"' {
			break
		} else if p.char == 0 {
			panic("Unexpected end of input in string")
		} else if rune(p.char) > unicode.MaxRune {
			panic(fmt.Sprintf("Invalid character code point %d in string", p.char))
		}
		p.next()
	}

	str := string(p.data[start:p.end])
	p.next()

	return str
}

func (p *JsonParser) parseMinus() *Node {
	start := p.end - 1
	p.next()
	switch p.char {
	case '0', '1', '2', '3', '4', '5', '6', '7', '8', '9':
		return p.parseNumber(start)
	}
	if !p.strict {

View on GitHub (pinned to 4f31cd3a0c)

Solutions

Transcode the input to valid UTF-8 before parsing (e.g. golang.org/x/text/encoding transform from the actual source encoding)
Sanitize with strings.ToValidUTF8(s, "\uFFFD") prior to Parse
Fix the producer to emit UTF-8 (set charset headers / file encodings correctly)
Remove embedded binary data from JSON; use base64 encoding for binary payloads

Example fix

// before
node := jsonx.Parse(rawLatin1Bytes, true)
// after
utf8Bytes, _ := charmap.ISO8859_1.NewDecoder().Bytes(rawLatin1Bytes)
node := jsonx.Parse(utf8Bytes, true)

Defensive patterns

Strategy: validation

Validate before calling

import "unicode/utf8"
if !utf8.Valid(b) {
	return errors.New("input is not valid UTF-8; transcode before parsing")
}

Try / catch

defer func() {
	if r := recover(); r != nil {
		if msg, ok := r.(string); ok && strings.Contains(msg, "Invalid character code point") {
			err = errors.New("invalid UTF-8 in input; sanitize with strings.ToValidUTF8")
		}
	}
}()

Prevention

Run utf8.Valid on all external input before parsing
Declare and honor charset headers; transcode latin-1/GBK sources
Base64-encode binary data instead of embedding raw bytes in JSON
Standardize producer output on UTF-8

When it happens

Trigger: Parse on input containing corrupted multi-byte UTF-8 sequences inside a string (e.g. a byte >= 0xF5 or broken continuation) such as `"\xFF\xFF"`-style binary data embedded in the JSON.

Common situations: Binary files accidentally parsed as JSON, latin-1/GBK encoded files read as UTF-8, database blobs pasted into payloads, charset mismatch between producer (ISO-8859-1) and consumer expecting UTF-8.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of antonmedv/fx@4f31cd3a0c (2026-09-02). Data as JSON: /api/errors/c56f305985a42268. Report an issue: GitHub.