{"record":{"id":"3d747f105826a7e4","repo":"jax-ml/jax","slug":"preemption-sync-manager-should-only-be-initialized","errorCode":null,"errorMessage":"Preemption sync manager should only be initialized once.","messagePattern":"Preemption sync manager should only be initialized once\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"jax/_src/distributed.py","lineNumber":258,"sourceCode":"      # client because the preemption sync manager depends on the client.\n      self.preemption_sync_manager.shutdown()\n      self.preemption_sync_manager = None\n    if self.client:\n      self.client.shutdown()\n      self.client = None\n    if self.service:\n      self.service.shutdown()\n      self.service = None\n\n  def initialize_preemption_sync_manager(self):\n    if not _ENABLE_PREEMPTION_SERVICE.value:\n      logger.info(\n          'The JAX preemption service is disabled. You can enable it using the'\n          ' jax_enable_preemption_service configuration option.'\n      )\n      return\n    if self.preemption_sync_manager is not None:\n      raise RuntimeError(\n          'Preemption sync manager should only be initialized once.')\n    self.preemption_sync_manager = (\n        _jax.create_preemption_sync_manager())\n    assert self.client is not None\n    self.preemption_sync_manager.initialize(self.client)\n\nglobal_state = State()\n\ndef initialize(coordinator_address: str | None = None,\n               num_processes: int | None = None,\n               process_id: int | None = None,\n               local_device_ids: int | Sequence[int] | None = None,\n               cluster_detection_method: str | None = None,\n               initialization_timeout: int = 300,\n               heartbeat_timeout_seconds: int = 100,\n               shutdown_timeout_seconds: int = 300,\n               coordinator_bind_address: str | None = None,\n               slice_index: int | None = None,","sourceCodeStart":240,"sourceCodeEnd":276,"githubUrl":"https://github.com/jax-ml/jax/blob/1e1c6a8fc06dfcd1247076ec5cae4640cea5d7bb/jax/_src/distributed.py#L240-L276","documentation":"Raised when initialize_preemption_sync_manager() is called after a preemption sync manager already exists for this process. The preemption sync manager (used to coordinate graceful shutdown across distributed workers) is a singleton bound to the distributed client.","triggerScenarios":"Calling jax.distributed.initialize() with preemption service enabled twice, or calling the internal initialize_preemption_sync_manager() a second time in one process.","commonSituations":"Re-running distributed setup in notebooks or test harnesses; combining a launcher that enables jax_enable_preemption_service with user code that also initializes it.","solutions":["Initialize distributed/preemption only once per process; remove duplicate calls","Restart the process/kernel to reset the singleton state","Check `global_state.preemption_sync_manager is None` before initializing"],"exampleFix":"// before\ninitialize()  # preemption enabled\ninitialize()  # RuntimeError\n\n// after\nfrom jax._src.distributed import global_state\nif global_state.preemption_sync_manager is None:\n    initialize()","handlingStrategy":"validation","validationCode":"from jax._src.distributed import global_state\nif global_state.preemption_sync_manager is None:\n    initialize_preemption_sync_manager()","typeGuard":null,"tryCatchPattern":"try:\n    initialize_preemption_sync_manager()\nexcept RuntimeError as e:\n    if 'initialized once' not in str(e): raise","preventionTips":["Treat preemption manager as process-global; init alongside distributed client once"],"tags":["jax","distributed","preemption","singleton"],"backgroundTag":"double-initialization","analyzedSha":"1e1c6a8fc06dfcd1247076ec5cae4640cea5d7bb","analyzedAt":"2026-08-27T09:53:25.647Z","schemaVersion":2},"datasetVersion":"2026-08-27T13:17:12.746Z"}