p-e-w/heretic · error · ValueError

prompts must not be empty

Error message

prompts must not be empty

What it means

get_residuals_mean computes the mean of residual activations over a batch of prompts; with an empty list the mean is mathematically undefined (would return NaN or crash in torch.cat), so it validates up front and raises ValueError.

Solutions

1. Ensure the prompt list passed in is non-empty; check the source dataset/file
2. Add an early guard in your pipeline: if not prompts: raise or skip
3. Log the prompt-count before calling get_residuals_mean to catch empty inputs early

Example fix

// before
mean = model.get_residuals_mean(prompts)
// after
if not prompts:
    raise ValueError("No prompts loaded; check dataset file")
mean = model.get_residuals_mean(prompts)

if not prompts:
    raise ValueError("prompts is empty; check dataset loading/filtering before calling get_residuals_mean")

def has_prompts(prompts: list) -> bool:
    return isinstance(prompts, list) and len(prompts) > 0

try:
    mean = model.get_residuals_mean(prompts)
except ValueError:
    logger.warning("No prompts to compute residuals over; skipping")
    mean = None

When it happens

Trigger: Calling model.get_residuals_mean([]) — typically when the prompt dataset/filter produced no prompts, e.g. an empty sample set loaded from a file or all prompts filtered out upstream in run.

Common situations: An empty prompts JSON/JSONL file, a filter step that removed everything, or a dataset path pointing at the wrong (empty) file.

Related errors

• cannot be empty or whitespace
• Failed to load model with all configured dtypes.
• '.' is not allowed
• requires settings to be validated
• .settings must be an instance of

AI-assisted analysis of p-e-w/heretic@bedb94ef11 (2026-08-29). Data as JSON: /api/errors/4abf3666f115c977. Report an issue: GitHub.

Appendix: source

Thrown at src/heretic/model.py:765

            residuals = torch.clamp(residuals, -thresholds, thresholds)

        if self.settings.offload_outputs_to_cpu:
            residuals = residuals.cpu()
            empty_cache()

        return residuals

    def get_residuals_batched(self, prompts: list[Prompt]) -> Tensor:
        residuals = []

        for batch in batchify(prompts, self.settings.batch_size):
            residuals.append(self.get_residuals(batch))

        return torch.cat(residuals, dim=0)

    def get_residuals_mean(self, prompts: list[Prompt]) -> Tensor:
        if not prompts:
            raise ValueError("prompts must not be empty")

        running_sum = None
        total_count = 0

        for batch in batchify(prompts, self.settings.batch_size):
            batch_residuals = self.get_residuals(batch)

            # Accumulate in high precision on CPU to reduce peak VRAM usage.
            batch_sum = batch_residuals.sum(dim=0, dtype=torch.float64).cpu()

            if running_sum is None:
                running_sum = batch_sum
            else:
                running_sum += batch_sum

            total_count += batch_residuals.shape[0]

        assert running_sum is not None

View on GitHub (pinned to bedb94ef11)