What causes "internal buffer has grown too big" in gravitational/teleport?

Ensure the Reader's consumer continuously drains it (do not block the read loop on slow UI/terminal writes).

internal buffer has grown too big

Error message

internal buffer has grown too big

What it means

ErrTooMuchBufferedData is returned by escape.Reader when its internal buffer exceeds 10MB because the consumer is not reading fast enough or is entirely stuck. It doubles as a stuck-session watchdog: setErr tears down the reader so an unresponsive downstream cannot buffer unboundedly.

Source

Thrown at lib/client/escape/reader.go:44

	"sync"
)

const (
	readerBufferLimit = 10 * 1024 * 1024 // 10MB

	// Note: on a raw terminal, "\r\n" is needed to move a cursor to the start
	// of next line.
	helpText = "\r\ntsh escape characters:\r\n  ~? - display a list of escape characters\r\n  ~. - disconnect\r\n"
)

var (
	// ErrDisconnect is returned when the user has entered a disconnect
	// sequence, requesting connection to be interrupted.
	ErrDisconnect = errors.New("disconnect escape sequence detected")
	// ErrTooMuchBufferedData is returned when the Reader's internal buffer has
	// filled over 10MB. Either the consumer of Reader can't keep up with the
	// data or it's entirely stuck and not consuming the data.
	ErrTooMuchBufferedData = errors.New("internal buffer has grown too big")
)

// Reader is an io.Reader wrapper that catches OpenSSH-like escape sequences in
// the input stream. See NewReader for more info.
//
// Reader is safe for concurrent use.
type Reader struct {
	inner        io.Reader
	out          io.Writer
	onDisconnect func(error)
	bufferLimit  int

	// cond protects buf and err and also announces to blocked readers that
	// more data is available.
	cond sync.Cond
	buf  []byte
	err  error
}

View on GitHub (pinned to 1283425b60)

Solutions

Ensure the Reader's consumer continuously drains it (do not block the read loop on slow UI/terminal writes).
Treat errors.Is(err, escape.ErrTooMuchBufferedData) as a stuck-session condition and terminate the session.
Investigate why the consumer stalled — check the writer side (terminal, logger, pipe) for blocking.
If data volume is legitimately high, consume faster or use a backpressure-aware design instead of buffering.

Example fix

// before
_, err := io.Copy(out, r)
require.Equal(t, err, ErrTooMuchBufferedData) // consumer stalled until 10MB filled
// after
// keep the consumer draining concurrently so the buffer never overflows
go func() { _, _ = io.Copy(out, r) }()

Defensive patterns

Strategy: type-guard

Type guard

func IsBufferOverflow(err error) bool { return errors.Is(err, escape.ErrTooMuchBufferedData) }

Try / catch

_, err := io.Copy(out, r)
if errors.Is(err, escape.ErrTooMuchBufferedData) {
    log.Warn("session consumer stalled; terminating stuck session")
    return errStuckSession
}

Prevention

Keep the reader's consumer draining continuously (never block the read loop on slow writers)
Monitor for consumer goroutine exits that leave the reader unconsumed
Treat this error as a stuck-session watchdog and terminate the session on it

When it happens

Trigger: reader.go:177 — runReads accumulated more than 10MB in the internal buffer while waiting for the consumer to Read; the reader unlocks, sets the error, and stops. Typically means the session output consumer stopped consuming (dead process, blocked pipe).

Common situations: A hung remote command producing output while the local terminal writer is blocked; a consumer goroutine that exited without closing the session; extremely bursty output exceeding 10MB faster than the terminal renders.

Related errors

AI-assisted analysis of gravitational/teleport@1283425b60 (2026-09-02). Data as JSON: /api/errors/72b86ea9007d031b. Report an issue: GitHub.