{"record":{"id":"8c97c1252a47aaef","repo":"vitessio/vitess","slug":"unicodeloosemd5-verify-v","errorCode":null,"errorMessage":"UnicodeLooseMD5.Verify: %v","messagePattern":"UnicodeLooseMD5\\.Verify: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"go/vt/vtgate/vindexes/unicodeloosemd5.go","lineNumber":78,"sourceCode":"}\n\n// IsUnique returns true since the Vindex is unique.\nfunc (vind *UnicodeLooseMD5) IsUnique() bool {\n\treturn true\n}\n\n// NeedsVCursor satisfies the Vindex interface.\nfunc (vind *UnicodeLooseMD5) NeedsVCursor() bool {\n\treturn false\n}\n\n// Verify returns true if ids maps to ksids.\nfunc (vind *UnicodeLooseMD5) Verify(ctx context.Context, vcursor VCursor, ids []sqltypes.Value, ksids [][]byte) ([]bool, error) {\n\tout := make([]bool, 0, len(ids))\n\tfor i, id := range ids {\n\t\tdata, err := vind.Hash(id)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"UnicodeLooseMD5.Verify: %v\", err)\n\t\t}\n\t\tout = append(out, bytes.Equal(data, ksids[i]))\n\t}\n\treturn out, nil\n}\n\n// Map can map ids to key.ShardDestination objects.\nfunc (vind *UnicodeLooseMD5) Map(ctx context.Context, vcursor VCursor, ids []sqltypes.Value) ([]key.ShardDestination, error) {\n\tout := make([]key.ShardDestination, 0, len(ids))\n\tfor _, id := range ids {\n\t\tdata, err := vind.Hash(id)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"UnicodeLooseMD5.Map: %v\", err)\n\t\t}\n\t\tout = append(out, key.DestinationKeyspaceID(data))\n\t}\n\treturn out, nil\n}","sourceCodeStart":60,"sourceCodeEnd":96,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/vtgate/vindexes/unicodeloosemd5.go#L60-L96","documentation":"UnicodeLooseMD5.Verify hashes each incoming id (via collation-normalized MD5) and compares it with the supplied keyspace ids; the vindex wraps any failure from vind.Hash in 'UnicodeLooseMD5.Verify: %v'. Hashing only fails when the id value cannot be converted to bytes or contains invalid UTF-8, which the collator cannot normalize. So this error means one of the rows passed to Verify had a non-string or non-UTF-8 value for the vindex column.","triggerScenarios":"Calling vindex Verify (directly or via a routed DML/SELECT where VTGate must verify ownership) with an ids[i] whose sqltypes.Value cannot ToBytes() (e.g. tuple/expression value) or whose bytes are not valid UTF-8.","commonSituations":"A vindex column defined over a VARBINARY/binary column so values carry invalid UTF-8; application inserts binary blobs or wrong-charset (e.g. latin1) data into a unicode_loose_md5 vindexed column; passing sqltypes.Value built from non-string bind vars in tests/tools.","solutions":["Inspect the id values in the failing statement; find rows with invalid UTF-8 (e.g. SELECT ... WHERE NOT JSON_TYPE ... or app-side utf8.Valid check).","Ensure the column backing the vindex uses a utf8/utf8mb4 charset so clients cannot store invalid UTF-8.","Convert the column charset (ALTER TABLE ... CONVERT TO CHARACTER SET utf8mb4) and re-map rows with bad data.","If the vindex column semantically holds binary data, switch to a binary vindex (binary, binary_md5, hash) instead of unicode_loose_md5.","If Hash fails because a value is not convertible to bytes, fix the caller (e.g. don't pass tuple values) to pass simple string values."],"exampleFix":"// before: vindex column VARBINARY receives latin1 data\nname VARBINARY(64), ... VINDEX = unicode_loose_md5\n// after\nALTER TABLE t MODIFY name VARCHAR(64) CHARACTER SET utf8mb4; -- VINDEX = unicode_loose_md5","handlingStrategy":"validation","validationCode":"if !utf8.Valid([]byte(val)) || val == \"\" {\n    // reject before issuing the routed DML\n}\n// ensure column charset: SHOW FULL COLUMNS FROM t; → utf8mb4","typeGuard":null,"tryCatchPattern":"res, err := vindex.Verify(ctx, vc, ids, ksids)\nif err != nil {\n    if strings.Contains(err.Error(), \"invalid UTF-8\") {\n        // re-encode or quarantine the offending ids\n    }\n    return err\n}","preventionTips":["Enforce utf8mb4 charset on vindex columns and connections","Validate UTF-8 in the application before writing vindexed fields","Use binary vindexes for non-text data","Never map blobs/binary columns to unicode_* vindexes"],"tags":["vtgate","vindex","utf-8","data-validation"],"backgroundTag":"invalid-utf8-vindex-value","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}