p-e-w/heretic · error · ValueError

{self.class.name} requires settings to be validated

Error message

{self.__class__.__name__} requires settings to be validated

What it means

The plugin class declares a settings schema (get_settings_model() is not None), but Plugin.__init__ was called with settings=None. The library requires that schema-declaring plugins always receive validated settings, so it raises ValueError rather than running with unvalidated/absent config.

Source

Thrown at src/heretic/plugin.py:221

        state outside the pinned config, for example:
        - It calls an external service (e.g. an LLM judge over the OpenAI API).
        - It reads credentials or config from the environment (env vars, files).
        - It is otherwise non-deterministic (network, wall-clock, unseeded RNG).

        Defaults to False; override to True in your plugin class if any of the
        above DO NOT apply.
        """
        return False

    def __init__(
        self, *, heretic_settings: HereticSettings, settings: BaseModel | None = None
    ):
        # Plugins that declare a settings schema should always receive
        # validated plugin settings from the evaluator.
        settings_model = self.__class__.get_settings_model()
        if settings_model is not None:
            if settings is None:
                raise ValueError(
                    f"{self.__class__.__name__} requires settings to be validated"
                )
            if not isinstance(settings, settings_model):
                raise TypeError(
                    f"{self.__class__.__name__}.settings must be an instance of "
                    f"{settings_model.__name__}"
                )
        self.settings = settings
        self.heretic_settings = heretic_settings

    @classmethod
    def validate_contract(cls) -> None:
        """
        Validate the plugin contract.

        - Plugins must not define a constructor (`__init__`). Initialization is
          handled by `Plugin.__init__` and an optional `init(ctx)` method.
        - Plugin subclasses may define `settings: <BaseModelSubclass>` to declare a settings schema.

View on GitHub (pinned to bedb94ef11)

Solutions

Pass a validated settings instance of the plugin's settings model to the constructor.
Add the plugin's namespace table to your TOML config so the evaluator builds and validates settings automatically.
If the plugin truly needs no settings, remove the `settings: <Model>` annotation from the class so get_settings_model() returns None.

Example fix

# before
plugin = MyScorer(heretic_settings=hs)
# after
plugin = MyScorer(heretic_settings=hs, settings=MyScorerSettings())

Defensive patterns

Strategy: validation

Validate before calling

model = MyScorer.get_settings_model()
assert model is not None, "plugin declares settings; supply validated settings"
settings = model(**cfg)
plugin = MyScorer(heretic_settings=hs, settings=settings)

Try / catch

try:
    plugin = MyScorer(heretic_settings=hs, settings=settings)
except ValueError as e:
    sys.exit(f"{e} — add the plugin's [settings] table to your config")

Prevention

Never construct schema-declaring plugins without settings.
Keep the plugin's TOML namespace table present in your config.
If settings are optional, remove the settings annotation instead of passing None.

When it happens

Trigger: Instantiating a plugin subclass directly as MyPlugin(heretic_settings=s) without passing settings; a custom evaluator/driver that bypasses the normal settings-validation path; the settings table for the plugin namespace is missing from the TOML config so None was forwarded.

Common situations: Hand-written test harness constructing the plugin manually; plugin recently gained a settings annotation while old callers still construct it without settings; config file lacking the [plugin_namespace] table.

Related errors

AI-assisted analysis of p-e-w/heretic@bedb94ef11 (2026-08-29). Data as JSON: /api/errors/7a32dac534fdf252. Report an issue: GitHub.