JuliusBrussee/caveman · error · ReplayRunError

cachebench: request %q failed model-visible equivalence

Error message

cachebench: request %q failed model-visible equivalence

What it means

Preparation verified that the optimised request body is not model-visible equivalent to the original — the optimisation changed bytes the model would see (a correctness violation), so the run fails closed with failure code model_visible_mismatch.

Source

Thrown at cacheengine/cachebench/replay.go:522

		if err := ctx.Err(); err != nil {
			return nil, fmt.Errorf("cachebench: replay preparation interrupted: %w", err)
		}
		native, err := record.NativeRequest()
		if err != nil {
			return nil, err
		}
		optimized, err := runner.Engine.Optimize(ctx, native)
		if err != nil {
			return nil, fmt.Errorf("cachebench: optimize request %q: %w", record.RequestID, err)
		}
		equivalent := bytes.Equal(native.Body, optimized.Body)
		if optimized.Applied {
			equivalent = ModelVisibleEquivalent(native.Body, optimized.Body)
		}
		if !equivalent {
			return nil, &ReplayRunError{
				RequestID: record.RequestID, FailureCode: "model_visible_mismatch",
				Err: fmt.Errorf("cachebench: request %q failed model-visible equivalence", record.RequestID),
			}
		}
		if optimized.Decision != cacheengine.DecisionApply && optimized.Decision != cacheengine.DecisionObserveOnly && optimized.Reason != cacheengine.ReasonBelowMinimum {
			return nil, &ReplayRunError{
				RequestID: record.RequestID, FailureCode: "engine_not_cacheable",
				Err: fmt.Errorf("cachebench: request %q is not cacheable: %s", record.RequestID, optimized.Reason),
			}
		}
		prepared = append(prepared, preparedReplay{record: record, optimized: optimized})
	}
	return prepared, nil
}

func validatePreparedReplayTarget(prepared []preparedReplay, target Target) error {
	if err := validateTarget(target); err != nil {
		return err
	}
	providers := map[string]bool{}

View on GitHub (pinned to 766dce6b13)

Solutions

1. Report the equivalence break to the engine maintainers with the offending request
2. Disable or fix the optimizer that altered model-visible content

When it happens

Trigger: Thrown at cacheengine/cachebench/replay.go:522 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of JuliusBrussee/caveman@766dce6b13 (2026-08-18). Data as JSON: /api/errors/ebc847dcabc30710. Report an issue: GitHub.