{"record":{"id":"f9441bcca3c6c7c0","repo":"Lightning-AI/pytorch-lightning","slug":"cannot-determine-global-rank-environment-variable","errorCode":null,"errorMessage":"Cannot determine global rank. Environment variable `JSM_NAMESPACE_RANK` not found. Make sure you run your executable with `jsrun`.","messagePattern":"Cannot determine global rank\\. Environment variable `JSM_NAMESPACE_RANK` not found\\. Make sure you run your executable with `jsrun`\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/lightning/fabric/plugins/environments/lsf.py","lineNumber":111,"sourceCode":"        \"\"\"The world size is read from the environment variable ``JSM_NAMESPACE_SIZE``.\"\"\"\n        world_size = os.environ.get(\"JSM_NAMESPACE_SIZE\")\n        if world_size is None:\n            raise ValueError(\n                \"Cannot determine world size. Environment variable `JSM_NAMESPACE_SIZE` not found.\"\n                \" Make sure you run your executable with `jsrun`.\"\n            )\n        return int(world_size)\n\n    @override\n    def set_world_size(self, size: int) -> None:\n        log.debug(\"LSFEnvironment.set_world_size was called, but setting world size is not allowed. Ignored.\")\n\n    @override\n    def global_rank(self) -> int:\n        \"\"\"The world size is read from the environment variable ``JSM_NAMESPACE_RANK``.\"\"\"\n        global_rank = os.environ.get(\"JSM_NAMESPACE_RANK\")\n        if global_rank is None:\n            raise ValueError(\n                \"Cannot determine global rank. Environment variable `JSM_NAMESPACE_RANK` not found.\"\n                \" Make sure you run your executable with `jsrun`.\"\n            )\n        return int(global_rank)\n\n    @override\n    def set_global_rank(self, rank: int) -> None:\n        log.debug(\"LSFEnvironment.set_global_rank was called, but setting global rank is not allowed. Ignored.\")\n\n    @override\n    def local_rank(self) -> int:\n        \"\"\"The local rank is read from the environment variable `JSM_NAMESPACE_LOCAL_RANK`.\"\"\"\n        local_rank = os.environ.get(\"JSM_NAMESPACE_LOCAL_RANK\")\n        if local_rank is None:\n            raise ValueError(\n                \"Cannot determine local rank. Environment variable `JSM_NAMESPACE_LOCAL_RANK` not found.\"\n                \" Make sure you run your executable with `jsrun`.\"\n            )","sourceCodeStart":93,"sourceCodeEnd":129,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/fabric/plugins/environments/lsf.py#L93-L129","documentation":"`LSFEnvironment.global_rank()` reads `JSM_NAMESPACE_RANK`, exported only when the process is launched with jsrun under IBM Spectrum Launch. Missing variable means the code is not running inside a jsrun-launched LSF job.","triggerScenarios":"Calling `global_rank` (directly or through Fabric/Trainer startup with LSFEnvironment) when `JSM_NAMESPACE_RANK` is unset — plain python launch, wrong scheduler, or an env-stripping wrapper.","commonSituations":"Same as other LSF errors: running outside jsrun on LSF clusters, wrapper scripts that don't propagate the environment, debugging rank logic locally.","solutions":["Run via jsrun so JSM_NAMESPACE_RANK is set: `jsrun ... python train.py`","If not on LSF, configure a different cluster environment matching your launcher","Sanity-check env inside the job: `echo $JSM_NAMESPACE_RANK`"],"exampleFix":"# before\npython train.py  # no JSM_NAMESPACE_RANK\n\n# after\njsrun -n 4 -a 1 python train.py  # sets JSM_NAMESPACE_RANK per process","handlingStrategy":"validation","validationCode":"import os\nassert os.environ.get('JSM_NAMESPACE_RANK') is not None, 'not running under jsrun'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Use jsrun to launch; don't run rank-dependent code on login nodes","Propagate full environment in wrapper scripts"],"tags":["lsf","jsrun","hpc","missing-env-var","distributed"],"backgroundTag":"missing-env-var","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}