cachebench: successful replay result is not bound to retaine

Error message

cachebench: successful replay result is not bound to retained artifacts

What it means

Error "cachebench: successful replay result is not bound to retained artifacts" thrown in JuliusBrussee/caveman.

Source

Thrown at cacheengine/cachebench/replay.go:831

	encoder := json.NewEncoder(writer)
	encoder.SetEscapeHTML(false)
	return encoder.Encode(record)
}

func emitValidatedReplayResult(emit func(ReplayResult) error, result ReplayResult) error {
	if err := validateReplayResult(result); err != nil {
		return err
	}
	return emit(result)
}

func validateReplayResult(result ReplayResult) error {
	if err := validateReplayEvidence(result.Evidence); err != nil {
		return err
	}
	if result.Evidence.Success {
		if result.Observation == nil || len(result.ProviderResponse) == 0 || len(result.VerificationEvidence) == 0 || bodyDigest(result.ProviderResponse) != result.Evidence.ProviderEvidenceSHA256 || bodyDigest(result.VerificationEvidence) != result.Evidence.QualityEvidenceSHA256 {
			return errors.New("cachebench: successful replay result is not bound to retained artifacts")
		}
		if err := validateObservationRecords([]ObservationRecord{*result.Observation}); err != nil {
			return err
		}
		observation := result.Observation
		if observation.RequestID != result.Evidence.RequestID || observation.Provider != result.Evidence.Provider || observation.Epoch != result.Evidence.Epoch || observation.RequestBodySHA256 != result.Evidence.TraceBodySHA256 || observation.ProviderEvidenceSHA256 != result.Evidence.ProviderEvidenceSHA256 || observation.EligibleInputTokens != result.Evidence.ProviderTotalInputTokens || observation.Applied != result.Evidence.Applied || observation.EngineDecision != result.Evidence.Decision || observation.EngineReason != result.Evidence.Reason || observation.Attribution != result.Evidence.Attribution || !slices.Equal(observation.OptimizerIDs, result.Evidence.OptimizerIDs) || observation.QualityPassed != result.Evidence.QualityPassed || observation.QualityEvidenceSHA256 != result.Evidence.QualityEvidenceSHA256 || observation.QualityVerifier != result.Evidence.QualityVerifier || bodyDigest(observation.Usage) != result.Evidence.ProviderUsageSHA256 {
			return errors.New("cachebench: replay observation is not bound to replay evidence")
		}
		return nil
	}
	if result.Observation != nil || len(result.VerificationEvidence) != 0 {
		return errors.New("cachebench: failed replay result contains success-only artifacts")
	}
	if len(result.ProviderResponse) > 0 {
		if bodyDigest(result.ProviderResponse) != result.Evidence.ProviderEvidenceSHA256 {
			return errors.New("cachebench: failed replay response is not bound to evidence")
		}
	} else if result.Evidence.ProviderEvidenceSHA256 != "" {

View on GitHub (pinned to 27d5a3981a)

Solutions

Bind the successful replay result to its retained artifacts.

When it happens

Trigger: Thrown at cacheengine/cachebench/replay.go:831 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/e1256a58bfcfbb51. Report an issue: GitHub.