sgl-project/sglang · error · ValueError

Number of labels must match the number of tag keys. Expected

Error message

Number of labels must match the number of tag keys. Expected {expected}, got {len(labels)}

What it means

When calling .labels(*labels) on a RayPrometheusMetric, positional label values must fill every tag key except the trailing ReplicaId, which the wrapper injects automatically. Supplying a different count of positional args is rejected.

Source

Thrown at python/sglang/srt/observability/ray_wrappers.py:108

                "RayPrometheusMetric requires Ray to be installed. "
                "Install with: pip install 'ray[serve]'"
            )
        self.metric: Optional[Metric] = None
        self._tags: dict = {"ReplicaId": _get_replica_id() or ""}

    @staticmethod
    def _get_tag_keys(labelnames: Optional[List[str]]) -> tuple:
        labels = list(labelnames) if labelnames else []
        labels.append("ReplicaId")
        return tuple(labels)

    def _build_tags(self, *labels: str, **labelskwargs: str) -> dict:
        if labels:
            # The trailing entry of ``_tag_keys`` is always ``ReplicaId`` which we
            # populate ourselves; positional args fill the preceding keys only.
            expected = len(self.metric._tag_keys) - 1
            if len(labels) != expected:
                raise ValueError(
                    "Number of labels must match the number of tag keys. "
                    f"Expected {expected}, got {len(labels)}"
                )
            labelskwargs.update(zip(self.metric._tag_keys, labels))
        labelskwargs["ReplicaId"] = _get_replica_id() or ""
        return {k: v if isinstance(v, str) else str(v) for k, v in labelskwargs.items()}

    def labels(self, *labels: str, **labelskwargs: str) -> RayPrometheusMetric:
        if self._is_labeled:
            raise ValueError("labels() cannot be called on an already-labeled metric.")
        clone = copy.copy(self)
        clone._tags = self._build_tags(*labels, **labelskwargs)
        clone._is_labeled = True
        return clone

    @staticmethod
    def _coerce_positive_boundaries(buckets):
        # Ray (gRPC OpenCensus / OpenTelemetry export) rejects boundaries

View on GitHub (pinned to 0132848349)

Solutions

Pass exactly len(tag_keys)-1 positional values (ReplicaId is auto-filled)
Prefer keyword labels matching tag key names to make arity explicit
Update all .labels() call sites whenever tag keys change

Example fix

# before
m = metric.labels('model')  # metric has 3 tag keys
# after
m = metric.labels('model', 'gpu_id')  # ReplicaId auto-injected

Defensive patterns

Strategy: validation

Validate before calling

expected = len(metric.metric._tag_keys) - 1
assert len(label_values) == expected, f'need {expected} positional labels'

Prevention

Use keyword labels matching tag key names
Define tag keys in one constant and generate label call sites from it

When it happens

Trigger: metric.labels('a') on a metric declared with 3 tag keys (expected=2); mixing positional and the wrong number of kwargs.

Common situations: Copy-pasting labels() calls between metrics with different tag signatures; adding a tag key to the metric definition without updating call sites.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b9d1e9d19c45b444. Report an issue: GitHub.