{"record":{"id":"e6a0a8553343957c","repo":"JuliusBrussee/caveman","slug":"session-value-action-s-lacks-promotion-evidence","errorCode":null,"errorMessage":"session-value action %s lacks promotion evidence","messagePattern":"session-value action (.+?) lacks promotion evidence","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"proxy/routing/session_value.go","lineNumber":36,"sourceCode":"\tSessionValueRouterVersion                = \"session-value-v1\"\n\tSessionValuePromotionMinEffectiveSamples = 100\n\tSessionValuePromotionMaxCalibrationError = 0.10\n\tSessionValuePromotionMaxBrierScore       = 0.25\n)\n\n// ValidateSessionValuePromotionEvidence applies evidence floors beyond\n// structural artifact validity. Runtime may evaluate a structurally valid\n// shadow artifact; traffic promotion requires calibrated sample floors.\nfunc ValidateSessionValuePromotionEvidence(artifact SessionValuePolicyArtifact) error {\n\tif len(artifact.Actions) == 0 {\n\t\treturn errors.New(\"session-value promotion artifact has no actions\")\n\t}\n\tfor _, action := range artifact.Actions {\n\t\tif action.RewardEffectiveSampleSize < SessionValuePromotionMinEffectiveSamples ||\n\t\t\taction.CostEffectiveSampleSize < SessionValuePromotionMinEffectiveSamples ||\n\t\t\taction.QualityCalibrationError > SessionValuePromotionMaxCalibrationError ||\n\t\t\taction.RewardBrierScore > SessionValuePromotionMaxBrierScore {\n\t\t\treturn fmt.Errorf(\"session-value action %s lacks promotion evidence\", action.ActionID)\n\t\t}\n\t}\n\treturn nil\n}\n\nvar sessionValueFeatureVocabulary = map[string]struct{}{\n\t\"turn_index\": {}, \"user_turns\": {}, \"subagent_turns\": {}, \"tool_calls\": {},\n\t\"corrective_turns\": {}, \"input_tokens_estimate\": {}, \"tools_count\": {},\n\t\"stream\": {}, \"json_mode\": {}, \"vision\": {}, \"audio\": {},\n\t\"cache_read_tokens\": {}, \"cache_write_tokens\": {}, \"uncached_tokens\": {},\n\t\"cache_age_ms\": {}, \"spent_session_usd\": {}, \"budget_remaining_usd\": {},\n\t\"output_token_prior\": {}, \"recent_error_count\": {}, \"interrupted_recently\": {},\n\t\"complexity_ordinal\": {},\n}\n\ntype SessionValueFeatureSpec struct {\n\tName     string  `json:\"name\"`\n\tMean     float64 `json:\"mean\"`","sourceCodeStart":18,"sourceCodeEnd":54,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/766dce6b1394ebb56a3090748d5a0240a5aefb36/proxy/routing/session_value.go#L18-L54","documentation":"ValidateSessionValuePromotionEvidence found an action in the shadow-mode policy artifact whose promotion evidence is missing or below the calibrated floors (minimum effective samples, max calibration error, max Brier score). Structurally valid artifacts may run in shadow; promoting an action to live traffic additionally requires this statistical evidence, and the action fails it.","triggerScenarios":"Thrown at proxy/routing/session_value.go:36 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":["Keep the action in shadow mode until it accrues at least the minimum effective samples with acceptable calibration error and Brier score","Regenerate the artifact after more data collection so the action's evidence meets the promotion floors"],"exampleFix":null,"handlingStrategy":"validation","validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"766dce6b1394ebb56a3090748d5a0240a5aefb36","analyzedAt":"2026-08-18T03:14:35.516Z","contentChangedAt":"2026-08-18T03:14:35.516Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}