JuliusBrussee/caveman · error

session-value action %s lacks promotion evidence

Error message

session-value action %s lacks promotion evidence

What it means

ValidateSessionValuePromotionEvidence found an action in the shadow-mode policy artifact whose promotion evidence is missing or below the calibrated floors (minimum effective samples, max calibration error, max Brier score). Structurally valid artifacts may run in shadow; promoting an action to live traffic additionally requires this statistical evidence, and the action fails it.

Source

Thrown at proxy/routing/session_value.go:36

	SessionValueRouterVersion                = "session-value-v1"
	SessionValuePromotionMinEffectiveSamples = 100
	SessionValuePromotionMaxCalibrationError = 0.10
	SessionValuePromotionMaxBrierScore       = 0.25
)

// ValidateSessionValuePromotionEvidence applies evidence floors beyond
// structural artifact validity. Runtime may evaluate a structurally valid
// shadow artifact; traffic promotion requires calibrated sample floors.
func ValidateSessionValuePromotionEvidence(artifact SessionValuePolicyArtifact) error {
	if len(artifact.Actions) == 0 {
		return errors.New("session-value promotion artifact has no actions")
	}
	for _, action := range artifact.Actions {
		if action.RewardEffectiveSampleSize < SessionValuePromotionMinEffectiveSamples ||
			action.CostEffectiveSampleSize < SessionValuePromotionMinEffectiveSamples ||
			action.QualityCalibrationError > SessionValuePromotionMaxCalibrationError ||
			action.RewardBrierScore > SessionValuePromotionMaxBrierScore {
			return fmt.Errorf("session-value action %s lacks promotion evidence", action.ActionID)
		}
	}
	return nil
}

var sessionValueFeatureVocabulary = map[string]struct{}{
	"turn_index": {}, "user_turns": {}, "subagent_turns": {}, "tool_calls": {},
	"corrective_turns": {}, "input_tokens_estimate": {}, "tools_count": {},
	"stream": {}, "json_mode": {}, "vision": {}, "audio": {},
	"cache_read_tokens": {}, "cache_write_tokens": {}, "uncached_tokens": {},
	"cache_age_ms": {}, "spent_session_usd": {}, "budget_remaining_usd": {},
	"output_token_prior": {}, "recent_error_count": {}, "interrupted_recently": {},
	"complexity_ordinal": {},
}

type SessionValueFeatureSpec struct {
	Name     string  `json:"name"`
	Mean     float64 `json:"mean"`

View on GitHub (pinned to 766dce6b13)

Solutions

1. Keep the action in shadow mode until it accrues at least the minimum effective samples with acceptable calibration error and Brier score
2. Regenerate the artifact after more data collection so the action's evidence meets the promotion floors

When it happens

Trigger: Thrown at proxy/routing/session_value.go:36 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of JuliusBrussee/caveman@766dce6b13 (2026-08-18). Data as JSON: /api/errors/e6a0a8553343957c. Report an issue: GitHub.