jackc/pgx · error

invalid tsvector binary format: lexeme

Error message

invalid tsvector binary format: lexeme %d: %w

What it means

While decoding lexeme %d of a binary tsvector, the reader errored (e.g. truncated lexeme word or bad position count). The lexeme index and wrapped reader error identify exactly where the payload broke.

Solutions

1. Fail the scan; the binary tsvector is truncated or corrupt
2. Inspect the wrapped error for the precise read failure
3. Check for proxies/custom encoders producing invalid tsvector bytes

When it happens

Trigger: Thrown at pgtype/tsvector.go:211 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of jackc/pgx@ec1a0befd2 (2026-08-04). Data as JSON: /api/errors/492357f19cbfccce. Report an issue: GitHub.

Appendix: source

Thrown at pgtype/tsvector.go:211

	// Each lexeme carries at minimum a 1-byte NUL terminator and a 2-byte position count. This
	// bounds the up-front make() against a malicious server claiming a huge lexeme count in a
	// small message.
	entryCount := r.Count(3)
	if err := r.Err(); err != nil {
		return fmt.Errorf("tsvector: %w", err)
	}

	var tsv TSVector
	if entryCount > 0 {
		tsv.Lexemes = make([]TSVectorLexeme, entryCount)
	}

	for i := range entryCount {
		lexeme := TSVectorLexeme{Word: string(r.CString())}

		numPositions := int(r.Uint16())
		if err := r.Err(); err != nil {
			return fmt.Errorf("invalid tsvector binary format: lexeme %d: %w", i, err)
		}

		// Each packed position is weight (2 bits) | position (14 bits). numPositions came from a
		// uint16, so it cannot ask for an unreasonable allocation here.
		if numPositions > 0 {
			lexeme.Positions = make([]TSVectorPosition, numPositions)
			for pos := range numPositions {
				packed := r.Uint16()
				lexeme.Positions[pos] = TSVectorPosition{
					Position: packed & 0x3FFF,
					Weight:   tsvectorWeightFromBinary(packed >> 14),
				}
			}
			if err := r.Err(); err != nil {
				return fmt.Errorf("invalid tsvector binary format: lexeme %d positions: %w", i, err)
			}
		}

View on GitHub (pinned to ec1a0befd2)