What causes "cachebench: nil corpus reader" in JuliusBrussee/caveman?

Ensure the reader comes from a successful open: `f, err := os.Open(path); if err != nil { return err }; defer f.Close()` before calling ReadAgentCorpus.

JuliusBrussee/caveman · error

cachebench: nil corpus reader

Error message

cachebench: nil corpus reader

What it means

Returned by ReadAgentCorpus() when the io.Reader passed in is nil. The importer is the entry point for converting JSONL corpora into the benchmark's internal representation; a nil reader would panic on the first Read, so it is rejected up front with a namespaced error ('cachebench:') before any metadata validation or format dispatch happens.

Source

Thrown at cacheengine/cachebench/corpus.go:137

}

type lmcacheRowWire struct {
	SessionID    string          `json:"session_id"`
	Model        string          `json:"model"`
	Input        json.RawMessage `json:"input"`
	OutputLength int             `json:"output_length"`
	PreGap       float64         `json:"pre_gap"`
}

type hfRowWire struct {
	RowIndex int            `json:"row_idx"`
	Row      lmcacheRowWire `json:"row"`
}

// ReadAgentCorpus imports supported JSONL formats under explicit limits.
func ReadAgentCorpus(reader io.Reader, format string, metadata CorpusMetadata, limits CorpusLimits) (AgentCorpus, error) {
	if reader == nil {
		return AgentCorpus{}, errors.New("cachebench: nil corpus reader")
	}
	if !validBoundedText(metadata.Name, 512, false) || !validBoundedText(metadata.License, 256, true) || !validBoundedText(metadata.Revision, 512, true) {
		return AgentCorpus{}, errors.New("cachebench: invalid corpus metadata")
	}
	var err error
	limits, err = normalizedCorpusLimits(limits)
	if err != nil {
		return AgentCorpus{}, err
	}
	limited := &io.LimitedReader{R: reader, N: limits.MaxInputBytes + 1}
	var rows []CorpusRow
	switch format {
	case CorpusFormatLMCacheJSONL:
		rows, err = readLMCacheJSONL(limited, limits)
	case CorpusFormatHFRows:
		rows, err = readHFRows(limited, limits)
	default:
		return AgentCorpus{}, fmt.Errorf("cachebench: unsupported corpus format %q", format)

View on GitHub (pinned to 27d5a3981a)

Solutions

Ensure the reader comes from a successful open: `f, err := os.Open(path); if err != nil { return err }; defer f.Close()` before calling ReadAgentCorpus.
If the input is genuinely optional, skip the call entirely rather than passing nil.
In tests, open the fixture (os.Open(testdata/file.jsonl)) instead of passing a nil literal placeholder.

Example fix

// before
f, _ := os.Open(path) // error ignored; f may be nil
corpus, err := cachebench.ReadAgentCorpus(f, "hf", meta, limits)

// after
f, err := os.Open(path)
if err != nil { return err }
defer f.Close()
corpus, err := cachebench.ReadAgentCorpus(f, "hf", meta, limits)

Defensive patterns

Strategy: validation

Validate before calling

// Go: never pass a nil reader
func openCorpus(path string) (io.Reader, error) {
    f, err := os.Open(path)
    if err != nil {
        return nil, err
    }
    return f, nil // caller: defer f.Close()
}

Type guard

func nonNilReader(r io.Reader) (io.Reader, bool) {
    return r, r != nil
}

Prevention

Always check os.Open's error and return it — the classic nil *os.File bug produces exactly this error.
Skip optional corpus imports rather than passing nil readers.
In tests, open real fixture files; never stub the reader with nil.

When it happens

Trigger: Calling ReadAgentCorpus(nil, ...) directly; a loader that returns a nil *os.File (failed os.Open whose error was swallowed) and passes it through; a test constructing the call without opening a fixture file.

Common situations: Go's classic os.Open error-shadowing bug (err captured but file still used); optional-input plumbing where a missing path becomes nil; wiring a new corpus source that forgets to open its file.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/c81a684bae1a2346. Report an issue: GitHub.