vitessio/vitess · error

malformed hex literal from parser

Error message

malformed hex literal from parser

What it means

parseHexNumber converts hex number literals ('0x...') produced by the SQL parser into binary bytes. It asserts the input actually starts with '0x'; if not, the parser and evalengine are inconsistent, which is an internal bug, so it panics rather than returning an error.

Source

Thrown at go/vt/vtgate/evalengine/api_literal.go:146

func NewLiteralDatetimeFromBytes(val []byte) (*Literal, error) {
	t, err := parseDateTime(val)
	if err != nil {
		return nil, err
	}
	return &Literal{t}, nil
}

func parseHexLiteral(val []byte) ([]byte, error) {
	raw := make([]byte, hex.DecodedLen(val))
	if err := hex.DecodeBytes(raw, val); err != nil {
		return nil, err
	}
	return raw, nil
}

func parseHexNumber(val []byte) ([]byte, error) {
	if val[0] != '0' || val[1] != 'x' {
		panic("malformed hex literal from parser")
	}
	if len(val)%2 == 0 {
		return parseHexLiteral(val[2:])
	}
	// If the hex literal doesn't have an even amount of hex digits, we need
	// to pad it with a '0' in the left. Instead of allocating a new slice
	// for padding pad in-place by replacing the 'x' in the original slice with
	// a '0', and clean it up after parsing.
	val[1] = '0'
	defer func() {
		val[1] = 'x'
	}()
	return parseHexLiteral(val[1:])
}

func parseBitNum(val []byte) ([]byte, error) {
	if val[0] != '0' || val[1] != 'b' {
		return nil, vterrors.Errorf(vtrpc.Code_INVALID_ARGUMENT, "malformed Bit literal: %q (missing 0b prefix)", val)

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Verify the input token starts with '0x' before calling NewLiteralBinaryFromHexNum (use parseHexLiteral for bare hex digits)
2. Regenerate the parser (make codegen) if lexer/parser rules were changed
3. Check for version skew between sqlparser and evalengine packages in your build

Example fix

// before
lit, err := NewLiteralBinaryFromHexNum([]byte("ff"))
// after
if len(b) >= 2 && b[0] == '0' && b[1] == 'x' {
    lit, err = NewLiteralBinaryFromHexNum(b)
} else {
    lit, err = NewLiteralBinaryFromHexNum([]byte("0x" + string(b)))
}

func isHexNum(b []byte) bool {
    return len(b) >= 3 && b[0] == '0' && b[1] == 'x'
}

func isHexNumberToken(val []byte) bool {
    return len(val) >= 3 && val[0] == '0' && val[1] == 'x'
}

defer func() {
    if r := recover(); r != nil {
        if strings.Contains(fmt.Sprint(r), "malformed hex literal") {
            log.Errorf("bad hex token: %q", tokenBytes)
            return
        }
        panic(r)
    }
}()

When it happens

Trigger: Calling NewLiteralBinaryFromHexNum (directly or via push_hexnum/valueToEval) with a token that does not begin with the two bytes '0','x' — i.e. the parser classified the token as a hex number but the bytes say otherwise.

Common situations: Custom parser forks or modified lexer rules emitting hexnum tokens for non-hex text; hand-crafted calls to NewLiteralBinaryFromHexNum in tests with raw byte slices lacking the 0x prefix; version mismatches between parser and evalengine packages.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• bad unsigned integer type
• bad type aggregation for signed/unsigned types
• unreachable
• unhandled case: evalIsTruthy
• EvalResult.coerce with no collation

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/81aa3ae3337ae6c4. Report an issue: GitHub.