{"record":{"id":"e72a9db32aca3650","repo":"sgl-project/sglang","slug":"labels-cannot-be-called-on-an-already-labeled-me","errorCode":null,"errorMessage":"labels() cannot be called on an already-labeled metric.","messagePattern":"labels\\(\\) cannot be called on an already-labeled metric\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/observability/ray_wrappers.py","lineNumber":118,"sourceCode":"        return tuple(labels)\n\n    def _build_tags(self, *labels: str, **labelskwargs: str) -> dict:\n        if labels:\n            # The trailing entry of ``_tag_keys`` is always ``ReplicaId`` which we\n            # populate ourselves; positional args fill the preceding keys only.\n            expected = len(self.metric._tag_keys) - 1\n            if len(labels) != expected:\n                raise ValueError(\n                    \"Number of labels must match the number of tag keys. \"\n                    f\"Expected {expected}, got {len(labels)}\"\n                )\n            labelskwargs.update(zip(self.metric._tag_keys, labels))\n        labelskwargs[\"ReplicaId\"] = _get_replica_id() or \"\"\n        return {k: v if isinstance(v, str) else str(v) for k, v in labelskwargs.items()}\n\n    def labels(self, *labels: str, **labelskwargs: str) -> RayPrometheusMetric:\n        if self._is_labeled:\n            raise ValueError(\"labels() cannot be called on an already-labeled metric.\")\n        clone = copy.copy(self)\n        clone._tags = self._build_tags(*labels, **labelskwargs)\n        clone._is_labeled = True\n        return clone\n\n    @staticmethod\n    def _coerce_positive_boundaries(buckets):\n        # Ray (gRPC OpenCensus / OpenTelemetry export) rejects boundaries\n        # <= 0. sglang ships several histograms whose lowest bucket is 0.0\n        # (e.g. queue_time, e2e latency). Silently drop those so we never\n        # break engine startup when the metrics backend is Ray.\n        if not buckets:\n            return []\n        return [b for b in buckets if b > 0]\n\n    @staticmethod\n    def _get_sanitized_opentelemetry_name(name: str) -> str:\n        \"\"\"Replace characters Ray's OTel-backed metric name validator rejects.","sourceCodeStart":100,"sourceCodeEnd":136,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/observability/ray_wrappers.py#L100-L136","documentation":"RayPrometheusMetric.labels() is designed to be called once on a fresh (unlabeled) metric; it returns a labeled clone and marks it. Calling .labels() again on that clone would stack/duplicate labels and is rejected.","triggerScenarios":"Chaining metric.labels(...).labels(...) or reusing a metric object that a factory already labeled.","commonSituations":"Shared metric singletons that get labeled at startup then again per-request; refactors that moved a labels() call into a helper invoked twice.","solutions":["Call labels() once on the original unlabeled metric and reuse the returned child for .observe/.inc etc.","Keep a reference to the unlabeled parent if you need differently-labeled children","Restructure helper functions so they return the labeled child instead of re-labeling"],"exampleFix":"# before\nchild = parent.labels('m1')\nchild = child.labels('m1')  # ValueError\n# after\nchild = parent.labels('m1')\nchild.inc()","handlingStrategy":"validation","validationCode":"assert not metric._is_labeled, 'already labeled; reuse the labeled child'\nlabeled = metric.labels('model', 'gpu')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Call labels() exactly once per metric instance; cache the labeled child","Have helpers return labeled children instead of re-labeling"],"tags":["observability","ray","metrics","api-misuse"],"backgroundTag":"double-label-metric","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}