What causes "session-value artifact candidate pool mismatch" in JuliusBrussee/caveman?

Recompute the live candidate pool hash (over the same CandidateActionID set the router uses) and regenerate the artifact whenever the roster changes.

JuliusBrussee/caveman · error

session-value artifact candidate pool mismatch

Error message

session-value artifact candidate pool mismatch

What it means

The artifact embeds CandidatePoolHash, a compact lowercase-hex hash of the exact candidate pool (the provider/model/effort set) it was trained on. Validation requires the hash to be well-formed (validCompactPoolHash: lowercase hex, length-validated) AND identical to the live pool hash passed in by the router — a policy trained on different candidates would score actions that no longer exist or miss new ones.

Source

Thrown at proxy/routing/session_value.go:175

		return SessionValuePolicyArtifact{}, err
	}
	artifact.ArtifactHash = hash
	return artifact, nil
}

func ValidateSessionValueArtifact(artifact SessionValuePolicyArtifact, organizationID, projectID, candidatePoolHash string, now time.Time) error {
	if artifact.Schema != SessionValueArtifactSchema || artifact.StateSchemaVersion != SessionValueStateSchema {
		return errors.New("session-value artifact schema mismatch")
	}
	if strings.TrimSpace(artifact.OrganizationID) == "" || artifact.OrganizationID != strings.TrimSpace(organizationID) ||
		strings.TrimSpace(artifact.ProjectID) == "" || artifact.ProjectID != strings.TrimSpace(projectID) {
		return errors.New("session-value artifact tenant scope mismatch")
	}
	if artifact.PolicyVersion <= 0 || artifact.RouterVersion != SessionValueRouterVersion || strings.TrimSpace(artifact.EstimatorVersion) == "" {
		return errors.New("session-value artifact version identity invalid")
	}
	if !validCompactPoolHash(artifact.CandidatePoolHash) || artifact.CandidatePoolHash != candidatePoolHash {
		return errors.New("session-value artifact candidate pool mismatch")
	}
	if !validSHA256Ref(artifact.TrainingManifestHash) || strings.TrimSpace(artifact.TrainingExtractor) == "" || strings.TrimSpace(artifact.OutcomeContractVersion) == "" {
		return errors.New("session-value artifact training lineage invalid")
	}
	if artifact.ValidFrom.IsZero() || artifact.ValidUntil.IsZero() || !artifact.ValidUntil.After(artifact.ValidFrom) || now.Before(artifact.ValidFrom) || !now.Before(artifact.ValidUntil) {
		return errors.New("session-value artifact outside validity window")
	}
	if artifact.RollbackParentHash != "" && (!validSHA256Ref(artifact.RollbackParentHash) || artifact.RollbackParentHash == artifact.ArtifactHash) {
		return errors.New("session-value artifact rollback lineage invalid")
	}
	if !finite(artifact.QualityUncertaintyZ) || artifact.QualityUncertaintyZ <= 0 || artifact.QualityUncertaintyZ > 5 ||
		!finite(artifact.MaxInversePropensity) || artifact.MaxInversePropensity < 1 || artifact.MaxInversePropensity > 100 {
		return errors.New("session-value artifact confidence policy invalid")
	}
	featureNames := SessionValueFeatureNames()
	if len(artifact.FeatureSpecs) != len(featureNames) || len(artifact.Actions) == 0 {
		return errors.New("session-value artifact has no features or actions")
	}

View on GitHub (pinned to 766dce6b13)

Solutions

Recompute the live candidate pool hash (over the same CandidateActionID set the router uses) and regenerate the artifact whenever the roster changes.
Gate deployment on pool-hash equality between router config and artifact so drift fails the pipeline, not production.
Check the hash format: compact lowercase hex exactly as validCompactPoolHash expects — no 'sha256:' prefix, no uppercase.

Example fix

// before
// artifact trained when pool was {claude-x, gpt-y}; gpt-z added to the router later
artifact.CandidatePoolHash = oldPoolHash

// after
// after any roster change, retrain and restamp
artifact.CandidatePoolHash = hashCurrentCandidatePool(routerCandidates())
artifact = routing.SealSessionValueArtifact(artifact) // re-seal so ArtifactHash covers the new pool

Defensive patterns

Strategy: validation

Validate before calling

// Before validation: recompute the live pool hash the way the router does and compare.
livePoolHash := hashCandidatePool(routerCandidatePool())
if artifact.CandidatePoolHash != livePoolHash {
	return fmt.Errorf("artifact pool hash %s does not match live pool %s; retrain after roster changes",
		artifact.CandidatePoolHash, livePoolHash)
}

Prevention

Make candidate-roster changes trigger artifact retraining in the same change or pipeline.
Compute the pool hash over the same canonical CandidateActionID set on both trainer and router.
Use compact lowercase hex for pool hashes — the format check (validCompactPoolHash) is strict.

When it happens

Trigger: The candidate roster changed (model added/removed, effort tier changed) after training without regenerating the artifact; the hash recorded in uppercase or with a prefix/suffix that breaks validCompactPoolHash; an artifact built against a different environment's pool being promoted.

Common situations: Model roster updates shipped without retraining the session-value policy; promoting an artifact from a staging pool to a production pool; trainer and router computing the pool hash over differently-ordered or differently-formatted candidates.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@766dce6b13 (2026-08-18). Data as JSON: /api/errors/b9f44c2ae32494c7. Report an issue: GitHub.