vitessio/vitess · error

UnicodeLooseMD5.Verify: %v

Error message

UnicodeLooseMD5.Verify: %v

What it means

UnicodeLooseMD5.Verify hashes each incoming id (via collation-normalized MD5) and compares it with the supplied keyspace ids; the vindex wraps any failure from vind.Hash in 'UnicodeLooseMD5.Verify: %v'. Hashing only fails when the id value cannot be converted to bytes or contains invalid UTF-8, which the collator cannot normalize. So this error means one of the rows passed to Verify had a non-string or non-UTF-8 value for the vindex column.

Source

Thrown at go/vt/vtgate/vindexes/unicodeloosemd5.go:78

}

// IsUnique returns true since the Vindex is unique.
func (vind *UnicodeLooseMD5) IsUnique() bool {
	return true
}

// NeedsVCursor satisfies the Vindex interface.
func (vind *UnicodeLooseMD5) NeedsVCursor() bool {
	return false
}

// Verify returns true if ids maps to ksids.
func (vind *UnicodeLooseMD5) Verify(ctx context.Context, vcursor VCursor, ids []sqltypes.Value, ksids [][]byte) ([]bool, error) {
	out := make([]bool, 0, len(ids))
	for i, id := range ids {
		data, err := vind.Hash(id)
		if err != nil {
			return nil, fmt.Errorf("UnicodeLooseMD5.Verify: %v", err)
		}
		out = append(out, bytes.Equal(data, ksids[i]))
	}
	return out, nil
}

// Map can map ids to key.ShardDestination objects.
func (vind *UnicodeLooseMD5) Map(ctx context.Context, vcursor VCursor, ids []sqltypes.Value) ([]key.ShardDestination, error) {
	out := make([]key.ShardDestination, 0, len(ids))
	for _, id := range ids {
		data, err := vind.Hash(id)
		if err != nil {
			return nil, fmt.Errorf("UnicodeLooseMD5.Map: %v", err)
		}
		out = append(out, key.DestinationKeyspaceID(data))
	}
	return out, nil
}

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Inspect the id values in the failing statement; find rows with invalid UTF-8 (e.g. SELECT ... WHERE NOT JSON_TYPE ... or app-side utf8.Valid check).
2. Ensure the column backing the vindex uses a utf8/utf8mb4 charset so clients cannot store invalid UTF-8.
3. Convert the column charset (ALTER TABLE ... CONVERT TO CHARACTER SET utf8mb4) and re-map rows with bad data.
4. If the vindex column semantically holds binary data, switch to a binary vindex (binary, binary_md5, hash) instead of unicode_loose_md5.
5. If Hash fails because a value is not convertible to bytes, fix the caller (e.g. don't pass tuple values) to pass simple string values.

Example fix

// before: vindex column VARBINARY receives latin1 data
name VARBINARY(64), ... VINDEX = unicode_loose_md5
// after
ALTER TABLE t MODIFY name VARCHAR(64) CHARACTER SET utf8mb4; -- VINDEX = unicode_loose_md5

if !utf8.Valid([]byte(val)) || val == "" {
    // reject before issuing the routed DML
}
// ensure column charset: SHOW FULL COLUMNS FROM t; → utf8mb4

res, err := vindex.Verify(ctx, vc, ids, ksids)
if err != nil {
    if strings.Contains(err.Error(), "invalid UTF-8") {
        // re-encode or quarantine the offending ids
    }
    return err
}

When it happens

Trigger: Calling vindex Verify (directly or via a routed DML/SELECT where VTGate must verify ownership) with an ids[i] whose sqltypes.Value cannot ToBytes() (e.g. tuple/expression value) or whose bytes are not valid UTF-8.

Common situations: A vindex column defined over a VARBINARY/binary column so values carry invalid UTF-8; application inserts binary blobs or wrong-charset (e.g. latin1) data into a unicode_loose_md5 vindexed column; passing sqltypes.Value built from non-string bind vars in tests/tools.

Related errors

• UnicodeLooseMD5.Map: %v
• UnicodeLooseXXHash.Verify: %v
• UnicodeLooseXXHash.Map: %v
• Binary.ReverseMap: keyspaceId is nil
• cannot map vindex to unique keyspace id: %v

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/8c97c1252a47aaef. Report an issue: GitHub.