{"record":{"id":"4c64e3d233c2a7ba","repo":"microsoft/garnet","slug":"malformed-ids-idlen-sizeof-int-remainin","errorCode":null,"errorMessage":"Malformed ids, {idLen} + {sizeof(int)} > {remainingIds.Length}","messagePattern":"Malformed ids, (.+?) \\+ (.+?) > (.+?)","errorType":"exception","errorClass":"GarnetException","httpStatus":null,"severity":"error","filePath":"libs/server/Resp/Vector/VectorManager.cs","lineNumber":1178,"sourceCode":"            byte[] idWithNamespaceArr = null;\n\n            var attributesNextIx = 0;\n\n            Span<byte> attributeFull = stackalloc byte[32];\n            var attributeMem = SpanByteAndMemory.FromPinnedSpan(attributeFull);\n\n            try\n            {\n                Span<byte> idWithNamespace = stackalloc byte[128];\n\n                // TODO: we could scatter/gather this like MGET - doesn't matter when everything is in memory,\n                //       but if anything is on disk it'd help perf\n                for (var i = 0; i < numIds; i++)\n                {\n                    var idLen = BinaryPrimitives.ReadInt32LittleEndian(remainingIds);\n                    if (idLen + sizeof(int) > remainingIds.Length)\n                    {\n                        throw new GarnetException($\"Malformed ids, {idLen} + {sizeof(int)} > {remainingIds.Length}\");\n                    }\n\n                    var id = remainingIds.Slice(sizeof(int), idLen);\n\n                    // Make sure we've got enough space to query the element\n                    if (id.Length + 1 > idWithNamespace.Length)\n                    {\n                        if (idWithNamespaceArr != null)\n                        {\n                            idPin.Free();\n                            ArrayPool<byte>.Shared.Return(idWithNamespaceArr);\n                        }\n\n                        idWithNamespaceArr = ArrayPool<byte>.Shared.Rent(id.Length + 1);\n                        idPin = GCHandle.Alloc(idWithNamespaceArr, GCHandleType.Pinned);\n                        idWithNamespace = idWithNamespaceArr;\n                    }\n","sourceCodeStart":1160,"sourceCodeEnd":1196,"githubUrl":"https://github.com/microsoft/garnet/blob/951b0fc6838721f89d102c2bbe1b914e8d39d700/libs/server/Resp/Vector/VectorManager.cs#L1160-L1196","documentation":"Thrown by FetchVectorElementAttributes while iterating the length-prefixed id buffer returned by the DiskANN search service. Each entry is expected to be a 4-byte little-endian length followed by that many id bytes; if the declared idLen plus the 4-byte prefix exceeds the remaining buffer length, the buffer is considered malformed. Critically, this buffer (outputIds) is produced internally by the vector service, not supplied by the RESP client, so the throw signals an internal invariant violation or on-disk data corruption rather than bad client input.","triggerScenarios":"Any VSIMILARITY/VSEARCH-style call that requests attributes or filtering (includeAttributes || !filter.IsEmpty) and then calls FetchVectorElementAttributes at VectorManager.cs:940 or :1109, when the DiskANN Service.SearchElement produced an outputIds buffer whose declared count (numIds) does not match its serialized length-prefix layout.","commonSituations":"Corrupted or partially-written Vector Set data on disk (e.g. an unclean shutdown or a truncated hybrid log), a serialization/deserialization bug in the DiskANN service for a specific vector dimension/quantization, or memory pressure causing a truncated output buffer. It is not caused by malformed RESP arguments.","solutions":["Treat this as a likely data-corruption / internal-bug condition: capture the Vector Set key, the query, and the logs, and file an upstream issue.","Recreate the affected Vector Set (VDROP and rebuild via VADD) to eliminate corrupt on-disk state.","If it reproduces deterministically with a given dataset, try a different quantization (e.g. switch between Q8/BIN/FLOAT) to isolate a codec-specific bug.","Verify storage integrity: run with --recover from a known-good checkpoint and check the device/filesystem for errors."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    await client.VSimilaritySearchAsync(key, queryVector, k, withAttrs: true);\n} catch (GarnetException ex) when (ex.Message.StartsWith(\"Malformed ids\")) {\n    // Internal invariant / data corruption, not bad input. Do not retry unchanged.\n    logger.LogCritical(ex, \"Vector Set {Key} appears corrupt (malformed id buffer)\", key);\n    // Optionally quarantine the key and rebuild it.\n}","preventionTips":["Run with durable checkpoints/AOF so a corrupt Vector Set can be rebuilt from source data.","Keep the source vectors so you can VDROP + re-VADD a suspect set.","Report a reproducible case upstream; capture the key, query, and quantization setting."],"tags":["garnet","vector-set","data-corruption","diskann","dotnet"],"backgroundTag":null,"analyzedSha":"951b0fc6838721f89d102c2bbe1b914e8d39d700","analyzedAt":"2026-08-13T19:01:32.939Z","schemaVersion":2},"datasetVersion":"2026-08-13T19:17:28.613Z"}