p-e-w/heretic · error · ValueError

'.' is not allowed

Error message

'.' is not allowed

What it means

validate_instance_name forbids the '.' character because dots are used as namespace separators in instance keys/plugin paths. A name containing '.' would create ambiguous or colliding identifiers, so it is rejected explicitly.

Source

Thrown at src/heretic/config.py:137

    instance_name: str | None = Field(
        default=None,
        description=(
            "Optional name to distinguish multiple instances of the same plugin class. "
            "Instance-specific settings live under `[scorer.<ClassName>_<instance_name>]`."
        ),
    )

    @field_validator("instance_name")
    @classmethod
    def validate_instance_name(cls, value: str | None) -> str | None:
        if value is None:
            return value

        if not value.strip():
            raise ValueError("cannot be empty or whitespace")

        if "." in value:
            raise ValueError("'.' is not allowed")

        if any(char.isspace() for char in value):
            raise ValueError("whitespace is not allowed")

        return value


class BenchmarkSpecification(BaseModel):
    task: str = Field(
        description="Task ID of the benchmark in the Language Model Evaluation Harness."
    )

    name: str = Field(description="Name of the benchmark for presentation purposes.")

    description: str = Field(
        description="Description of the benchmark for presentation purposes."
    )

View on GitHub (pinned to bedb94ef11)

Solutions

1. Replace '.' in the name with '_' or '-' (e.g. "my_scorer")
2. Derive the name programmatically: value.replace(".", "_") before validation
3. Use a flat, dot-free identifier in config.toml

Example fix

// before
instance_name = "scorers.toxicity"
// after
instance_name = "scorers_toxicity"

if "." in (instance_name or ""):
    raise ValueError("instance_name must not contain '.'")

def is_dot_free(name: str) -> bool:
    return isinstance(name, str) and "." not in name

try:
    cfg.validate_instance_name(name)
except ValueError:
    name = name.replace(".", "_")
    cfg.validate_instance_name(name)

When it happens

Trigger: Calling validate_instance_name with a value containing a period, e.g. "my.scorer", usually from dotted config keys, module-style names, or filenames used as instance names.

Common situations: Developers naming instances after Python modules ("scorers.toxicity"), Hugging Face repo-style ids ("org/model"), or config subsection paths copied verbatim.

Related errors

• whitespace is not allowed
• cannot be empty or whitespace
• Duplicate scorer instance name: {scorer_key}. Give each inst
• No scorers configured. Set 'scorers' in config.toml
• prompts must not be empty

AI-assisted analysis of p-e-w/heretic@bedb94ef11 (2026-08-29). Data as JSON: /api/errors/0124beabf3224d8d. Report an issue: GitHub.