hankcs/HanLP · error · NotImplementedError

mask not supported in SpearmanCorrelation for now.

Error message

mask not supported in SpearmanCorrelation for now.

What it means

SpearmanCorrelation.__call__ rejects a non-None mask argument because mask-aware computation was never implemented for this metric. It raises NotImplemented (itself a bug: NotImplemented is a constant, not an exception, so it technically raises a TypeError at raise-time).

Source

Thrown at hanlp/metrics/spearman_correlation.py:69

        self.total_predictions = torch.zeros(0)
        self.total_gold_labels = torch.zeros(0)

    def __call__(
            self,
            predictions: torch.Tensor,
            gold_labels: torch.Tensor,
            mask=None
    ):
        """
        # Parameters

        predictions : `torch.Tensor`, required.
            A tensor of predictions of shape (batch_size, ...).
        gold_labels : `torch.Tensor`, required.
            A tensor of the same shape as `predictions`.
        """
        if mask is not None:
            raise NotImplemented('mask not supported in SpearmanCorrelation for now.')
        # Flatten predictions, gold_labels, and mask. We calculate the Spearman correlation between
        # the vectors, since each element in the predictions and gold_labels tensor is assumed
        # to be a separate observation.
        predictions = predictions.reshape(-1)
        gold_labels = gold_labels.reshape(-1)

        self.total_predictions = self.total_predictions.to(predictions.device)
        self.total_gold_labels = self.total_gold_labels.to(gold_labels.device)
        self.total_predictions = torch.cat((self.total_predictions, predictions), 0)
        self.total_gold_labels = torch.cat((self.total_gold_labels, gold_labels), 0)

    def reset(self):
        self.total_predictions = torch.zeros(0)
        self.total_gold_labels = torch.zeros(0)

    def __str__(self) -> str:
        return f'spearman: {self.score * 100:.2f}'

View on GitHub (pinned to ddb1299bdd)

Solutions

1. Drop the mask argument when calling SpearmanCorrelation; pre-filter predictions/gold_labels instead.
2. If masking is genuinely needed, compute it yourself with scipy.stats.spearmanr on the masked elements.
3. Note for maintainers: the check should raise NotImplementedError, not NotImplemented, otherwise you get a confusing TypeError.

Example fix

# before
metric(predictions, gold_labels, mask=mask)
# after
metric(predictions.reshape(-1)[mask.bool()], gold_labels.reshape(-1)[mask.bool()])

if mask is not None:
    predictions = predictions.reshape(-1)[mask.reshape(-1).bool()]
    gold_labels = gold_labels.reshape(-1)[mask.reshape(-1).bool()]
metric(predictions, gold_labels)

def call_metric(metric, pred, gold, mask=None):
    import inspect
    if mask is not None and 'SpearmanCorrelation' in type(metric).__name__:
        pred, gold = pred.reshape(-1)[mask.reshape(-1).bool()], gold.reshape(-1)[mask.reshape(-1).bool()]
        mask = None
    return metric(pred, gold, mask=mask)

try:
    metric(pred, gold)
except TypeError:
    metric(pred.reshape(-1), gold.reshape(-1))  # mask path unsupported

When it happens

Trigger: Passing mask=... to a SpearmanCorrelation Metric object, e.g. when a training loop uniformly forwards masks to all metrics including this one.

Common situations: Frameworks/allennlp-style training loops that always pass a mask; users copying usage from MaskedAverageAccuracy-style metrics; upgrading code where the metric signature gained a mask parameter.

Related errors

• Unrecognized label encoding {self.label_encoding}
• Unknown constraint type: {constraint_type}
• " ".join(tag_sequence)

AI-assisted analysis of hankcs/HanLP@ddb1299bdd (2026-08-27). Data as JSON: /api/errors/625c556097133590. Report an issue: GitHub.