{"record":{"id":"1fc9ca2073b887c7","repo":"p-e-w/heretic","slug":"duplicate-scorer-instance-name-scorer-key-give","errorCode":null,"errorMessage":"Duplicate scorer instance name: {scorer_key}. Give each instance a unique `instance_name`.","messagePattern":"Duplicate scorer instance name: (.+?)\\. Give each instance a unique `instance_name`\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/heretic/evaluator.py","lineNumber":90,"sourceCode":"            )\n            scorer_settings: BaseModel | None = scorer_cls.validate_settings(\n                raw_settings\n            )\n\n            scorer = scorer_cls(\n                heretic_settings=self.settings,\n                settings=scorer_settings,\n            )\n\n            # External labeling key: ensures multiple instances can coexist.\n            # Uses underscore to match the TOML namespace format (`scorer.<Class>_<instance>`).\n            scorer_key = (\n                scorer_cls.__name__\n                if not instance_name\n                else f\"{scorer_cls.__name__}_{instance_name}\"\n            )\n            if scorer_key in scorer_keys:\n                raise ValueError(\n                    f\"Duplicate scorer instance name: {scorer_key}. \"\n                    \"Give each instance a unique `instance_name`.\"\n                )\n            scorer_keys.add(scorer_key)\n\n            scorer_instance_name = (\n                f\"{scorer.score_name} - {instance_name}\"\n                if instance_name\n                else scorer.score_name\n            )\n            self._scorer_entries.append(\n                ScorerEntry(scorer=scorer, config=config, name=scorer_instance_name)\n            )\n\n        # Run scorer init hooks.\n        ctx = Context(settings=self.settings, model=self.model)\n\n        for entry in self._scorer_entries:","sourceCodeStart":72,"sourceCodeEnd":108,"githubUrl":"https://github.com/p-e-w/heretic/blob/bedb94ef117a271532ac2058447fbc165d5051bd/src/heretic/evaluator.py#L72-L108","documentation":"Each scorer instance gets a key of `<PluginClassName>_<instance_name>` (or just the class name if no instance_name). Two configurations resolving to the same key would silently overwrite each other's settings/state, so a duplicate key raises with instructions to set a unique instance_name.","triggerScenarios":"Configuring the same scorer plugin twice without distinct instance_name values (e.g. two [[scorers]] entries with plugin = \"refusal\" and no instance_name), or two entries whose class names and instance names collide.","commonSituations":"Running one scorer with different settings copies (forgetting to rename instances), refactoring a config and duplicating an entry, or copying an existing [[scorers]] block to add a variant.","solutions":["Give each duplicate plugin entry a distinct instance_name (e.g. \"strict\", \"lenient\")","Remove the redundant duplicate [[scorers]] entry if it was accidental","Key the key derivation by also distinguishing settings if you intend genuinely separate instances"],"exampleFix":"// before (config.toml)\n[[scorers]]\nplugin = \"refusal\"\n[[scorers]]\nplugin = \"refusal\"\n// after (config.toml)\n[[scorers]]\nplugin = \"refusal\"\ninstance_name = \"strict\"\n[[scorers]]\nplugin = \"refusal\"\ninstance_name = \"lenient\"\n","handlingStrategy":"validation","validationCode":"keys = [f\"{s.plugin}_{s.instance_name}\" if s.instance_name else s.plugin for s in settings.scorers]\nif len(keys) != len(set(keys)):\n    raise ValueError(f\"Duplicate scorer keys: {keys}\")","typeGuard":null,"tryCatchPattern":"try:\n    evaluator = Evaluator(settings)\nexcept ValueError as e:\n    if e.message.startswith(\"Duplicate scorer instance name\"):\n        dedupe_scorer_configs(settings)\n        evaluator = Evaluator(settings)\n    else:\n        raise","preventionTips":["Always set a unique instance_name when using a plugin more than once","Keep a naming convention (plugin + variant) for scorer instances","Lint config.toml for duplicate [[scorers]] plugin+name pairs"],"tags":["config","duplicate","naming"],"backgroundTag":"duplicate-identifier","analyzedSha":"bedb94ef117a271532ac2058447fbc165d5051bd","analyzedAt":"2026-08-29T08:38:06.692Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}