microsoft/garnet · error · GarnetException

Exceeded maximum response size of ({Array.MaxLength:N0}) byt

Error message

Exceeded maximum response size of ({Array.MaxLength:N0}) bytes

What it means

Thrown by RespMemoryWriter.ReallocateOutput when the computed new buffer length overflows to zero or negative after doubling or rounding up to a power of two. This is an overflow guard: the internal RESP output buffer cannot grow beyond Array.MaxLength (~2 GB). The GarnetException is thrown with disposeSession:false so the session survives. It indicates a single response (or accumulated batch) that exceeds the maximum representable buffer size.

Source

Thrown at libs/common/RespMemoryWriter.cs:524

            {
                // Maximal allocation from MemoryPool Rent()
                length = Array.MaxLength;
            }
            else if (length < extraLenHint)
            {
                var total = (uint)extraLenHint + (uint)length;
                if ((total >= 0x40000000) && (total < Array.MaxLength))
                    length = Array.MaxLength;
                else
                    length = (int)BitOperations.RoundUpToPowerOf2(total);
            }
            else
            {
                length <<= 1;
            }

            if (length <= 0)
                throw new GarnetException($"Exceeded maximum response size of ({Array.MaxLength:N0}) bytes", disposeSession: false);

            var newMem = MemoryPool<byte>.Shared.Rent(length);
            var newPtrHandle = newMem.Memory.Pin();
            var newPtr = (byte*)newPtrHandle.Pointer;
            var bytesWritten = (int)(curr - ptr);
            if (bytesWritten > 0)
                Buffer.MemoryCopy(ptr, newPtr, length, bytesWritten);

            if (ptrHandle.Pointer != default)
            {
                ptrHandle.Dispose();
                output.Memory.Dispose();
            }
            else
            {
                output.ConvertToHeap();
            }

View on GitHub (pinned to 951b0fc683)

Solutions

Paginate or chunk the large read (use SCAN + individual GETs, or LRANGE/HSCAN with bounded counts) instead of fetching everything at once.
If using a custom command, cap or stream the output rather than buffering the entire response.
Review the query that produced the error and add a LIMIT / COUNT bound.
If the data genuinely needs to be this large, use a dedicated dump/export path (e.g., RDB/checkpoint) rather than a single RESP response.
Check for a response-generation bug (infinite loop, wrong length) in custom command code.

Example fix

// before: fetch entire huge hash in one call
var all = db.HashGetAll("giant-hash");

// after: scan in bounded batches
var batch = new IBatch[]{};
await foreach (var entries in db.HashScanAsync("giant-hash", pageSize: 1000))
{
    Process(entries);
}

Defensive patterns

Strategy: validation

Validate before calling

// Before issuing a large read, estimate the result size and reject/page early.
long EstimateHashSize(IDatabase db, string key, long maxBytes)
{
    var len = db.HashLength(key); // number of fields
    // Assume ~avgFieldSize per entry; tune to your data
    const long avgFieldSize = 128;
    long estBytes = len * avgFieldSize;
    if (estBytes > maxBytes)
        throw new InvalidOperationException($"Hash '{key}' (~{estBytes} bytes) exceeds safe response size.");
    return estBytes;
}

Try / catch

try
{
    var result = db.HashGetAll("large-hash");
}
catch (GarnetException ex) when (ex.Message.Contains("Exceeded maximum response size"))
{
    // Fall back to paginated reads via HSCAN
    await foreach (var batch in db.HashScanAsync("large-hash", pageSize: 1000))
        Process(batch);
}

Prevention

Paginate large multi-element reads (HSCAN, SSCAN, LRANGE with bounds).
Cap response sizes in custom commands at the application layer.
Monitor response sizes for outlier keys and split or trim them.
Never issue an unbounded GETALL/HGETALL on keys of unknown cardinality.

When it happens

Trigger: Issuing a command whose response payload is extremely large — e.g., MGET / LRANGE / SORT / HGETALL on a very large keyspace, or a custom command returning a huge blob — causing the output writer to repeatedly double its buffer past int.MaxValue. Also triggered by a runaway response-generation bug producing an unbounded write.

Common situations: Querying a hash/list/set with millions of members in a single round-trip; a script or custom command returning an unbounded result; a serialization bug that writes far more than expected; exporting a full dataset snapshot via a single RESP response.

Related errors

AI-assisted analysis of microsoft/garnet@951b0fc683 (2026-08-13). Data as JSON: /api/errors/98860b9aaad6b056. Report an issue: GitHub.