{"record":{"id":"7df6be3610fdb193","repo":"sgl-project/sglang","slug":"kv-canary-launch-per-forward-not-supported-on-swe","errorCode":null,"errorMessage":"kv-canary: launch_per_forward not supported on sweep endpoint {self.kernel_kind.name}","messagePattern":"kv-canary: launch_per_forward not supported on sweep endpoint (.+?)","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/kv_canary/endpoint.py","lineNumber":55,"sourceCode":"    kernel_run_counter_view: torch.Tensor\n    enable_chain_position_assert: torch.Tensor\n\n    def launch_per_forward(\n        self,\n        *,\n        verify_plan: VerifyPlan,\n        write_plan: WritePlan,\n        input_ids: torch.Tensor,\n        positions: torch.Tensor,\n        out_cache_loc: torch.Tensor,\n        enable_write_input_assert: bool,\n        enable_verify_token_assert: bool,\n        expected_inputs: ExpectedInputs,\n        violation_log: ViolationLog,\n        real_kv_hash_mode: RealKvHashMode,\n    ) -> None:\n        if _is_sweep_tag(self.kernel_kind):\n            raise NotImplementedError(\n                f\"kv-canary: launch_per_forward not supported on sweep endpoint {self.kernel_kind.name}\"\n            )\n\n        context = self._make_verify_or_write_context(\n            violation_log=violation_log,\n            real_kv_hash_mode=real_kv_hash_mode,\n        )\n        launch_canary_verify_kernel(\n            context=context,\n            plan=verify_plan,\n            check_verify_expected_token=enable_verify_token_assert,\n        )\n\n        # SWA endpoints translate the per-token slot indices via a device tensor index op before invoking the write kernel.\n        if self.full_to_swa_index_mapping is not None:\n            out_cache_loc_for_canary = self.full_to_swa_index_mapping[out_cache_loc]\n        else:\n            out_cache_loc_for_canary = out_cache_loc","sourceCodeStart":37,"sourceCodeEnd":73,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/kv_canary/endpoint.py#L37-L73","documentation":"launch_per_forward is only implemented for single fixed kernels; canary endpoint objects whose kernel_kind carries a sweep tag run a sweep of kernels instead and cannot be driven per-forward. Calling launch_per_forward on such an endpoint raises NotImplementedError naming the sweep kind.","triggerScenarios":"Calling endpoint.launch_per_forward(...) (directly or via launch_endpoints_per_forward) on an endpoint constructed with a sweep kernel_kind (any kind matching the internal sweep tag) instead of a single kernel kind.","commonSituations":"Extending kv-canary with a new sweep-style kernel kind and routing it through the per-forward path; internal code changes that call per-forward launch on all endpoints indiscriminately instead of dispatching sweep endpoints to the sweep driver.","solutions":["Dispatch sweep-tagged endpoints to the sweep launch path, not launch_per_forward (check _is_sweep_tag(kernel_kind) before calling)","Use a single (non-sweep) kernel_kind for endpoints that must support per-forward launches","Guard the call site with _is_sweep_tag and skip/log for sweep endpoints"],"exampleFix":"# before\nfor ep in endpoints:\n    ep.launch_per_forward(ctx, ...)  # raises on sweep endpoints\n\n# after\nfor ep in endpoints:\n    if not _is_sweep_tag(ep.kernel_kind):\n        ep.launch_per_forward(ctx, ...)","handlingStrategy":"type-guard","validationCode":"from sglang.srt.kv_canary.endpoint import _is_sweep_tag\nfor ep in endpoints:\n    if _is_sweep_tag(ep.kernel_kind):\n        continue  # route sweep endpoints to the sweep driver instead\n    ep.launch_per_forward(ctx, ...)","typeGuard":"def supports_per_forward(ep) -> bool:\n    return not _is_sweep_tag(ep.kernel_kind)","tryCatchPattern":"try:\n    ep.launch_per_forward(...)\nexcept NotImplementedError as e:\n    if 'launch_per_forward not supported on sweep endpoint' in str(e):\n        run_sweep_instead(ep)\n    else:\n        raise","preventionTips":["Gate per-forward launches with _is_sweep_tag(kernel_kind)","Keep sweep endpoints on the sweep launch path in dispatch code","Add unit tests asserting sweep kinds raise on launch_per_forward so regressions in dispatch surface early"],"tags":["kv-canary","endpoint","sweep-kernel","not-implemented"],"backgroundTag":"unsupported-operation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}