{"record":{"id":"98860b9aaad6b056","repo":"microsoft/garnet","slug":"exceeded-maximum-response-size-of-array-maxlengt","errorCode":null,"errorMessage":"Exceeded maximum response size of ({Array.MaxLength:N0}) bytes","messagePattern":"Exceeded maximum response size of \\((.+?)\\) bytes","errorType":"exception","errorClass":"GarnetException","httpStatus":null,"severity":"error","filePath":"libs/common/RespMemoryWriter.cs","lineNumber":524,"sourceCode":"            {\n                // Maximal allocation from MemoryPool Rent()\n                length = Array.MaxLength;\n            }\n            else if (length < extraLenHint)\n            {\n                var total = (uint)extraLenHint + (uint)length;\n                if ((total >= 0x40000000) && (total < Array.MaxLength))\n                    length = Array.MaxLength;\n                else\n                    length = (int)BitOperations.RoundUpToPowerOf2(total);\n            }\n            else\n            {\n                length <<= 1;\n            }\n\n            if (length <= 0)\n                throw new GarnetException($\"Exceeded maximum response size of ({Array.MaxLength:N0}) bytes\", disposeSession: false);\n\n            var newMem = MemoryPool<byte>.Shared.Rent(length);\n            var newPtrHandle = newMem.Memory.Pin();\n            var newPtr = (byte*)newPtrHandle.Pointer;\n            var bytesWritten = (int)(curr - ptr);\n            if (bytesWritten > 0)\n                Buffer.MemoryCopy(ptr, newPtr, length, bytesWritten);\n\n            if (ptrHandle.Pointer != default)\n            {\n                ptrHandle.Dispose();\n                output.Memory.Dispose();\n            }\n            else\n            {\n                output.ConvertToHeap();\n            }\n","sourceCodeStart":506,"sourceCodeEnd":542,"githubUrl":"https://github.com/microsoft/garnet/blob/951b0fc6838721f89d102c2bbe1b914e8d39d700/libs/common/RespMemoryWriter.cs#L506-L542","documentation":"Thrown by RespMemoryWriter.ReallocateOutput when the computed new buffer length overflows to zero or negative after doubling or rounding up to a power of two. This is an overflow guard: the internal RESP output buffer cannot grow beyond Array.MaxLength (~2 GB). The GarnetException is thrown with disposeSession:false so the session survives. It indicates a single response (or accumulated batch) that exceeds the maximum representable buffer size.","triggerScenarios":"Issuing a command whose response payload is extremely large — e.g., MGET / LRANGE / SORT / HGETALL on a very large keyspace, or a custom command returning a huge blob — causing the output writer to repeatedly double its buffer past int.MaxValue. Also triggered by a runaway response-generation bug producing an unbounded write.","commonSituations":"Querying a hash/list/set with millions of members in a single round-trip; a script or custom command returning an unbounded result; a serialization bug that writes far more than expected; exporting a full dataset snapshot via a single RESP response.","solutions":["Paginate or chunk the large read (use SCAN + individual GETs, or LRANGE/HSCAN with bounded counts) instead of fetching everything at once.","If using a custom command, cap or stream the output rather than buffering the entire response.","Review the query that produced the error and add a LIMIT / COUNT bound.","If the data genuinely needs to be this large, use a dedicated dump/export path (e.g., RDB/checkpoint) rather than a single RESP response.","Check for a response-generation bug (infinite loop, wrong length) in custom command code."],"exampleFix":"// before: fetch entire huge hash in one call\nvar all = db.HashGetAll(\"giant-hash\");\n\n// after: scan in bounded batches\nvar batch = new IBatch[]{};\nawait foreach (var entries in db.HashScanAsync(\"giant-hash\", pageSize: 1000))\n{\n    Process(entries);\n}","handlingStrategy":"validation","validationCode":"// Before issuing a large read, estimate the result size and reject/page early.\nlong EstimateHashSize(IDatabase db, string key, long maxBytes)\n{\n    var len = db.HashLength(key); // number of fields\n    // Assume ~avgFieldSize per entry; tune to your data\n    const long avgFieldSize = 128;\n    long estBytes = len * avgFieldSize;\n    if (estBytes > maxBytes)\n        throw new InvalidOperationException($\"Hash '{key}' (~{estBytes} bytes) exceeds safe response size.\");\n    return estBytes;\n}","typeGuard":null,"tryCatchPattern":"try\n{\n    var result = db.HashGetAll(\"large-hash\");\n}\ncatch (GarnetException ex) when (ex.Message.Contains(\"Exceeded maximum response size\"))\n{\n    // Fall back to paginated reads via HSCAN\n    await foreach (var batch in db.HashScanAsync(\"large-hash\", pageSize: 1000))\n        Process(batch);\n}","preventionTips":["Paginate large multi-element reads (HSCAN, SSCAN, LRANGE with bounds).","Cap response sizes in custom commands at the application layer.","Monitor response sizes for outlier keys and split or trim them.","Never issue an unbounded GETALL/HGETALL on keys of unknown cardinality."],"tags":["memory","buffer","overflow","resp","response-size","respmemorywriter"],"backgroundTag":null,"analyzedSha":"951b0fc6838721f89d102c2bbe1b914e8d39d700","analyzedAt":"2026-08-13T19:01:32.939Z","schemaVersion":2},"datasetVersion":"2026-08-13T19:17:28.613Z"}