golang/go · error

bufio.Scanner: SplitFunc returns negative advance count

Error message

bufio.Scanner: SplitFunc returns negative advance count

What it means

bufio.Scanner returns ErrNegativeAdvance when a custom SplitFunc returns an advance value less than zero (scan.go:243-246 via Scanner.advance). The SplitFunc contract requires 0 <= advance <= len(data); a negative advance would move the scan pointer backwards, so the Scanner rejects it and stops. This signals a bug in the user-supplied split function, never in the built-in ScanLines/ScanWords/etc.

Source

Thrown at src/bufio/scan.go:72

//
// Otherwise, the [Scanner] advances the input. If the token is not nil,
// the [Scanner] returns it to the user. If the token is nil, the
// Scanner reads more data and continues scanning; if there is no more
// data--if atEOF was true--the [Scanner] returns. If the data does not
// yet hold a complete token, for instance if it has no newline while
// scanning lines, a [SplitFunc] can return (0, nil, nil) to signal the
// [Scanner] to read more data into the slice and try again with a
// longer slice starting at the same point in the input.
//
// The function is never called with an empty data slice unless atEOF
// is true. If atEOF is true, however, data may be non-empty and,
// as always, holds unprocessed text.
type SplitFunc func(data []byte, atEOF bool) (advance int, token []byte, err error)

// Errors returned by Scanner.
var (
	ErrTooLong         = errors.New("bufio.Scanner: token too long")
	ErrNegativeAdvance = errors.New("bufio.Scanner: SplitFunc returns negative advance count")
	ErrAdvanceTooFar   = errors.New("bufio.Scanner: SplitFunc returns advance count beyond input")
	ErrBadReadCount    = errors.New("bufio.Scanner: Read returned impossible count")
)

const (
	// MaxScanTokenSize is the maximum size used to buffer a token
	// unless the user provides an explicit buffer with [Scanner.Buffer].
	// The actual maximum token size may be smaller as the buffer
	// may need to include, for instance, a newline.
	MaxScanTokenSize = 64 * 1024

	startBufSize = 4096 // Size of initial allocation for buffer.
)

// NewScanner returns a new [Scanner] to read from r.
// The split function defaults to [ScanLines].
func NewScanner(r io.Reader) *Scanner {
	return &Scanner{

View on GitHub (pinned to b6b368adc5)

Solutions

1. Audit the custom SplitFunc: ensure the returned advance is always >= 0 and <= len(data).
2. Return (0, nil, nil) when you cannot make progress, instead of a negative advance, to ask the Scanner for more data.
3. Add a property test feeding random inputs and asserting the SplitFunc never returns a negative advance.
4. If you do not need custom splitting, remove the Scanner.Split call and use a built-in split function.

Example fix

// before — advance can go negative
split := func(data []byte, atEOF bool) (int, []byte, error) {
    i := bytes.IndexByte(data, '\t')
    return i - 1, data[:i], nil // negative if i == 0
}

// after — guard the advance
split := func(data []byte, atEOF bool) (int, []byte, error) {
    i := bytes.IndexByte(data, '\t')
    if i < 0 {
        if !atEOF { return 0, nil, nil }
        return len(data), data, nil
    }
    return i + 1, data[:i], nil
}

// Validate a custom SplitFunc's return before handing it to the Scanner.
func validateSplit(split bufio.SplitFunc) bufio.SplitFunc {
    return func(data []byte, atEOF bool) (int, []byte, error) {
        adv, tok, err := split(data, atEOF)
        if adv < 0 {
            return 0, nil, fmt.Errorf("split returned negative advance %d", adv)
        }
        if adv > len(data) {
            return 0, nil, fmt.Errorf("split returned advance %d > len(data) %d", adv, len(data))
        }
        return adv, tok, err
    }
}
// sc.Split(validateSplit(mySplit))

// Property: a well-formed SplitFunc never returns negative advance.
func splitIsSafe(split bufio.SplitFunc) bool {
    // exercise with sample inputs and confirm invariants; omitted for brevity
    return true
}

for sc.Scan() {
    process(sc.Bytes())
}
if err := sc.Err(); err != nil {
    if errors.Is(err, bufio.ErrNegativeAdvance) {
        // the custom SplitFunc has a bug; audit it
    }
}

When it happens

Trigger: Triggered only when a custom SplitFunc passed to Scanner.Split returns (advance<0, token, err). Reachable on every Scan() call that invokes the split function. The built-in split functions (ScanLines, ScanWords, ScanRunes, ScanBytes) never produce this.

Common situations: Hand-written SplitFunc that computes advance from an index subtraction that can go negative (e.g., advance = strings.Index(...) - base where the match precedes base). Porting a tokenizer from another language where negative advances were tolerated. Off-by-one in delimiter accounting.

Related errors

• bufio.Scanner: SplitFunc returns advance count beyond input
• bufio.Scanner: token too long
• final token
• bufio: reader returned negative count from Read
• bufio: writer returned negative count from Write

AI-assisted analysis of golang/go@b6b368adc5 (2026-08-12). Data as JSON: /api/errors/40a69bc74dc66697. Report an issue: GitHub.