What causes "cannot be empty or whitespace" in p-e-w/heretic?

Set a non-empty instance_name in config.toml or the API call (e.g. instance_name = "scorer-a")

p-e-w/heretic · error · ValueError

cannot be empty or whitespace

Error message

cannot be empty or whitespace

What it means

validate_instance_name rejects instance names that are empty or contain only whitespace characters. The library requires every scorer/instance name to be a non-empty, meaningful identifier so generated keys are unambiguous. It fails fast at config-validation time rather than producing broken instance keys later.

Source

Thrown at src/heretic/config.py:134

        ),
    )

    instance_name: str | None = Field(
        default=None,
        description=(
            "Optional name to distinguish multiple instances of the same plugin class. "
            "Instance-specific settings live under `[scorer.<ClassName>_<instance_name>]`."
        ),
    )

    @field_validator("instance_name")
    @classmethod
    def validate_instance_name(cls, value: str | None) -> str | None:
        if value is None:
            return value

        if not value.strip():
            raise ValueError("cannot be empty or whitespace")

        if "." in value:
            raise ValueError("'.' is not allowed")

        if any(char.isspace() for char in value):
            raise ValueError("whitespace is not allowed")

        return value


class BenchmarkSpecification(BaseModel):
    task: str = Field(
        description="Task ID of the benchmark in the Language Model Evaluation Harness."
    )

    name: str = Field(description="Name of the benchmark for presentation purposes.")

    description: str = Field(

View on GitHub (pinned to bedb94ef11)

Solutions

Set a non-empty instance_name in config.toml or the API call (e.g. instance_name = "scorer-a")
If the name is optional, omit the field entirely (pass None) instead of an empty string
Trim upstream input and skip/None-out empty values before validation

Example fix

// before
instance_name = ""
// after
instance_name = "scorer-a"

Defensive patterns

Strategy: validation

Validate before calling

if instance_name is not None and not instance_name.strip():
    raise ValueError("instance_name must not be empty or whitespace")

Type guard

def is_valid_instance_name(v: str | None) -> bool:
    return v is None or (bool(v.strip()) and "." not in v and not any(c.isspace() for c in v))

Try / catch

try:
    cfg.validate_instance_name(raw_name)
except ValueError as e:
    logger.error("Invalid instance_name: %s", e)
    raw_name = None  # fall back to default

Prevention

Omit instance_name rather than passing ""
Strip and normalize user input before validation
Add a config lint step that checks instance names

When it happens

Trigger: Calling HereticConfig.validate_instance_name with an empty string ("") or a string of only spaces/tabs (e.g. " "), typically via instance_name parsed from config.toml or CLI where the value was left blank.

Common situations: An unset-but-present `instance_name = ""` in config.toml, shell variable interpolation producing an empty value (`instance_name="$MY_VAR"` with MY_VAR unset), or copy-pasting a name with only spaces.

Related errors

AI-assisted analysis of p-e-w/heretic@bedb94ef11 (2026-08-29). Data as JSON: /api/errors/2b1326e293dc96ca. Report an issue: GitHub.