golang/go · critical

bytes.Buffer: too large

Error message

bytes.Buffer: too large

What it means

bytes.Buffer panics with ErrTooLarge when it cannot allocate enough memory to grow the internal slice (buffer.go:50, re-panicked from growSlice at buffer.go:247-252). It is also used to guard against integer overflow: the buffer refuses to grow beyond maxInt. Because allocation failure is fatal to the buffer's invariants, the package panics rather than returning an error. Callers that may write unbounded data should recover or cap input.

Source

Thrown at src/bytes/buffer.go:50

// The readOp constants describe the last action performed on
// the buffer, so that UnreadRune and UnreadByte can check for
// invalid usage. opReadRuneX constants are chosen such that
// converted to int they correspond to the rune size that was read.
type readOp int8

// Don't use iota for these, as the values need to correspond with the
// names and comments, which is easier to see when being explicit.
const (
	opRead      readOp = -1 // Any other read operation.
	opInvalid   readOp = 0  // Non-read operation.
	opReadRune1 readOp = 1  // Read rune of size 1.
	opReadRune2 readOp = 2  // Read rune of size 2.
	opReadRune3 readOp = 3  // Read rune of size 3.
	opReadRune4 readOp = 4  // Read rune of size 4.
)

// ErrTooLarge is passed to panic if memory cannot be allocated to store data in a buffer.
var ErrTooLarge = errors.New("bytes.Buffer: too large")
var errNegativeRead = errors.New("bytes.Buffer: reader returned negative count from Read")

const maxInt = int(^uint(0) >> 1)

// Bytes returns a slice of length b.Len() holding the unread portion of the buffer.
// The slice is valid for use only until the next buffer modification (that is,
// only until the next call to a method like [Buffer.Read], [Buffer.Write], [Buffer.Reset], or [Buffer.Truncate]).
// The slice aliases the buffer content at least until the next buffer modification,
// so immediate changes to the slice will affect the result of future reads.
func (b *Buffer) Bytes() []byte { return b.buf[b.off:] }

// AvailableBuffer returns an empty buffer with b.Available() capacity.
// This buffer is intended to be appended to and
// passed to an immediately succeeding [Buffer.Write] call.
// The buffer is only valid until the next write operation on b.
func (b *Buffer) AvailableBuffer() []byte { return b.buf[len(b.buf):] }

// String returns the contents of the unread portion of the buffer

View on GitHub (pinned to b6b368adc5)

Solutions

1. Cap the source before/while reading: limit with io.LimitReader, or check size before copying into the buffer.
2. If unbounded growth is legitimate, wrap buffer operations in a deferred recover() that treats ErrTooLarge as 'input too big'.
3. On 32-bit targets, switch to streaming (io.Copy to the final destination) instead of buffering the whole payload.
4. Raise container memory limits or switch to a chunked/file-backed buffer for very large payloads.

Example fix

// before — unbounded ReadFrom can panic with ErrTooLarge
buf := new(bytes.Buffer)
io.Copy(buf, resp.Body) // may OOM

// after — cap the input and recover as a safety net
buf := new(bytes.Buffer)
io.Copy(buf, io.LimitReader(resp.Body, maxBodyBytes))

// or, guard explicitly:
defer func() {
    if r := recover(); r != nil {
        if r == bytes.ErrTooLarge { /* handle oversized input */ }
        panic(r)
    }
}()

// Cap input size before buffering; optionally guard with recover.
buf := new(bytes.Buffer)
_, err := io.Copy(buf, io.LimitReader(src, maxBytes))
if err != nil { return err }
if buf.Len() >= maxBytes { return ErrTooBig }

defer func() {
    if r := recover(); r != nil {
        if r == bytes.ErrTooLarge {
            err = fmt.Errorf("input exceeded buffer capacity")
            return
        }
        panic(r)
    }
}()
_, err = buf.ReadFrom(src)

When it happens

Trigger: Triggered by any write path that calls grow/growSlice — Write, WriteString, WriteByte, WriteRune, ReadFrom — when make([]byte, newSize) panics (OOM) or the requested size overflows int. The recover in growSlice (buffer.go:248-251) converts any allocation panic into a re-panic carrying ErrTooLarge.

Common situations: Calling buffer.ReadFrom on an unbounded stream (network, decompression) with no size limit. Accumulating logs/responses into a Buffer that grows without bound. 32-bit builds where maxInt is ~2 GB and the buffer hits the ceiling sooner. Memory-constrained containers where make() fails at a few hundred MB.

Related errors

• bytes.Buffer: truncation out of range
• bytes.Buffer.Grow: negative count
• bytes.Buffer: UnreadRune: previous operation was not a succe
• bytes.Buffer: UnreadByte: previous operation was not a succe
• bytes: Join output length overflow

AI-assisted analysis of golang/go@b6b368adc5 (2026-08-12). Data as JSON: /api/errors/fc541332e340f1ed. Report an issue: GitHub.