golang/go · error

bufio.Scanner: SplitFunc returns advance count beyond input

Error message

bufio.Scanner: SplitFunc returns advance count beyond input

What it means

bufio.Scanner returns ErrAdvanceTooFar when a SplitFunc returns an advance greater than the number of available bytes in the current buffer window (scan.go:248-251 via Scanner.advance). The SplitFunc contract requires advance <= len(data slice passed in); returning a larger value would read past the buffer, so the Scanner treats it as a bug and stops. Only custom SplitFuncs produce this; the built-in ones are correct by construction.

Source

Thrown at src/bufio/scan.go:73

// Otherwise, the [Scanner] advances the input. If the token is not nil,
// the [Scanner] returns it to the user. If the token is nil, the
// Scanner reads more data and continues scanning; if there is no more
// data--if atEOF was true--the [Scanner] returns. If the data does not
// yet hold a complete token, for instance if it has no newline while
// scanning lines, a [SplitFunc] can return (0, nil, nil) to signal the
// [Scanner] to read more data into the slice and try again with a
// longer slice starting at the same point in the input.
//
// The function is never called with an empty data slice unless atEOF
// is true. If atEOF is true, however, data may be non-empty and,
// as always, holds unprocessed text.
type SplitFunc func(data []byte, atEOF bool) (advance int, token []byte, err error)

// Errors returned by Scanner.
var (
	ErrTooLong         = errors.New("bufio.Scanner: token too long")
	ErrNegativeAdvance = errors.New("bufio.Scanner: SplitFunc returns negative advance count")
	ErrAdvanceTooFar   = errors.New("bufio.Scanner: SplitFunc returns advance count beyond input")
	ErrBadReadCount    = errors.New("bufio.Scanner: Read returned impossible count")
)

const (
	// MaxScanTokenSize is the maximum size used to buffer a token
	// unless the user provides an explicit buffer with [Scanner.Buffer].
	// The actual maximum token size may be smaller as the buffer
	// may need to include, for instance, a newline.
	MaxScanTokenSize = 64 * 1024

	startBufSize = 4096 // Size of initial allocation for buffer.
)

// NewScanner returns a new [Scanner] to read from r.
// The split function defaults to [ScanLines].
func NewScanner(r io.Reader) *Scanner {
	return &Scanner{
		r:            r,

View on GitHub (pinned to b6b368adc5)

Solutions

1. Recompute advance so it never exceeds len(data) — the slice the SplitFunc received.
2. If you need bytes beyond the current window, return (0, nil, nil) to request more data (the Scanner will grow the buffer and re-invoke you).
3. Unit-test the SplitFunc against inputs where the delimiter straddles buffer boundaries.
4. Replace the custom SplitFunc with a built-in (ScanLines, ScanWords) if your splitting is line/word based.

Example fix

// before — advance overshoots when delimiter missing from chunk
split := func(data []byte, atEOF bool) (int, []byte, error) {
    // assumes a trailing newline always present
    return len(data) + 1, data[:len(data)-1], nil // ErrAdvanceTooFar
}

// after — respect the slice boundary
split := func(data []byte, atEOF bool) (int, []byte, error) {
    if atEOF && len(data) > 0 {
        return len(data), data, nil
    }
    return 0, nil, nil // ask for more
}

// Reuse the same validator wrapper as ErrNegativeAdvance — it also catches overshoot.
func validateSplit(split bufio.SplitFunc) bufio.SplitFunc {
    return func(data []byte, atEOF bool) (int, []byte, error) {
        adv, tok, err := split(data, atEOF)
        if adv < 0 || adv > len(data) {
            return 0, nil, fmt.Errorf("invalid advance %d for data len %d", adv, len(data))
        }
        return adv, tok, err
    }
}

for sc.Scan() {}
if err := sc.Err(); err != nil {
    if errors.Is(err, bufio.ErrAdvanceTooFar) {
        // custom SplitFunc over-advanced; fix it
    }
}

When it happens

Trigger: Triggered when s.split(...) returns advance > (s.end - s.start) for the current window, and Scanner.advance(n) detects it (scan.go:248-249). Happens when a SplitFunc returns len(data)+1 or advances based on a lookahead beyond the supplied slice.

Common situations: Custom SplitFunc that returns advance = len(data) + delimiterLen to skip a trailing delimiter that is not actually in `data`. Miscalculation when a delimiter is at the very end of the chunk and the function assumes an extra byte. Ported splitter that assumed access to the full input rather than the current window.

Related errors

• bufio.Scanner: SplitFunc returns negative advance count
• bufio.Scanner: token too long
• final token
• bufio: reader returned negative count from Read
• bufio: writer returned negative count from Write

AI-assisted analysis of golang/go@b6b368adc5 (2026-08-12). Data as JSON: /api/errors/f6cc29f4ec77e522. Report an issue: GitHub.