p-e-w/heretic · error · ValueError

prompts must not be empty

Error message

prompts must not be empty

What it means

get_residuals_mean computes the mean of residual activations over a batch of prompts; with an empty list the mean is mathematically undefined (would return NaN or crash in torch.cat), so it validates up front and raises ValueError.

Source

Thrown at src/heretic/model.py:765

            residuals = torch.clamp(residuals, -thresholds, thresholds)

        if self.settings.offload_outputs_to_cpu:
            residuals = residuals.cpu()
            empty_cache()

        return residuals

    def get_residuals_batched(self, prompts: list[Prompt]) -> Tensor:
        residuals = []

        for batch in batchify(prompts, self.settings.batch_size):
            residuals.append(self.get_residuals(batch))

        return torch.cat(residuals, dim=0)

    def get_residuals_mean(self, prompts: list[Prompt]) -> Tensor:
        if not prompts:
            raise ValueError("prompts must not be empty")

        running_sum = None
        total_count = 0

        for batch in batchify(prompts, self.settings.batch_size):
            batch_residuals = self.get_residuals(batch)

            # Accumulate in high precision on CPU to reduce peak VRAM usage.
            batch_sum = batch_residuals.sum(dim=0, dtype=torch.float64).cpu()

            if running_sum is None:
                running_sum = batch_sum
            else:
                running_sum += batch_sum

            total_count += batch_residuals.shape[0]

        assert running_sum is not None

View on GitHub (pinned to bedb94ef11)

Solutions

1. Ensure the prompt list passed in is non-empty; check the source dataset/file
2. Add an early guard in your pipeline: if not prompts: raise or skip
3. Log the prompt-count before calling get_residuals_mean to catch empty inputs early

Example fix

// before
mean = model.get_residuals_mean(prompts)
// after
if not prompts:
    raise ValueError("No prompts loaded; check dataset file")
mean = model.get_residuals_mean(prompts)

if not prompts:
    raise ValueError("prompts is empty; check dataset loading/filtering before calling get_residuals_mean")

def has_prompts(prompts: list) -> bool:
    return isinstance(prompts, list) and len(prompts) > 0

try:
    mean = model.get_residuals_mean(prompts)
except ValueError:
    logger.warning("No prompts to compute residuals over; skipping")
    mean = None

When it happens

Trigger: Calling model.get_residuals_mean([]) — typically when the prompt dataset/filter produced no prompts, e.g. an empty sample set loaded from a file or all prompts filtered out upstream in run.

Common situations: An empty prompts JSON/JSONL file, a filter step that removed everything, or a dataset path pointing at the wrong (empty) file.

Related errors

• cannot be empty or whitespace
• '.' is not allowed
• whitespace is not allowed
• Failed to load model with all configured dtypes.
• {self.__class__.__name__} requires settings to be validated

AI-assisted analysis of p-e-w/heretic@bedb94ef11 (2026-08-29). Data as JSON: /api/errors/4abf3666f115c977. Report an issue: GitHub.