{"record":{"id":"1f5c29514f710b26","repo":"apache/beam","slug":"session-file-found-but-empty-s-s","errorCode":null,"errorMessage":"Session file found, but empty: %s.%s","messagePattern":"Session file found, but empty: (.+?)\\.(.+?)","errorType":"exception","errorClass":"LoadMainSessionException","httpStatus":null,"severity":"critical","filePath":"sdks/python/apache_beam/runners/worker/sdk_worker_main.py","lineNumber":412,"sourceCode":"    warn_msg = ' Functions defined in __main__ (interactive session) may fail.'\n    err_msg = ' Functions defined in __main__ (interactive session) will ' \\\n      'almost certainly fail.'\n  elif pickler.is_currently_cloudpickle():\n    warn_msg = ' User registered objects (e.g. schema, logical type) through' \\\n        'registeries may not be effective'\n    err_msg = ''\n  if semi_persistent_directory:\n    session_file = os.path.join(\n        semi_persistent_directory, 'staged', names.PICKLED_MAIN_SESSION_FILE)\n    if os.path.isfile(session_file):\n      # If the expected session file is present but empty, it's likely that\n      # the user code run by this worker will likely crash at runtime.\n      # This can happen if the worker fails to download the main session.\n      # Raise a fatal error and crash this worker, forcing a restart.\n      if os.path.getsize(session_file) == 0:\n        if pickler.is_currently_dill():\n          # Potenitally transient error, unclear if still happening.\n          raise LoadMainSessionException(\n              'Session file found, but empty: %s.%s' % (session_file, err_msg))\n        else:\n          _LOGGER.warning('Empty session file: %s.%s', warn_msg, session_file)\n      else:\n        pickler.load_session(session_file)\n    else:\n      _LOGGER.warning('No session file found: %s.%s', warn_msg, session_file)\n  else:\n    _LOGGER.warning('No semi_persistent_directory found: %s', warn_msg)\n\n\nif __name__ == '__main__':\n  main(sys.argv)\n","sourceCodeStart":394,"sourceCodeEnd":426,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/worker/sdk_worker_main.py#L394-L426","documentation":"During worker startup, create_harness loads the pickled main session file containing the user's pipeline dependencies. If the session file exists but has zero bytes, the worker cannot restore user code; when running under dill (is_currently_dill) this is treated as fatal via LoadMainSessionException ('Session file found, but empty'), crashing the worker to force a restart and re-download.","triggerScenarios":"The staged session file (~templated or staged to the worker) downloads as 0 bytes — transient fetch failure — while the worker uses the dill pickler.","commonSituations":"Dataflow workers failing to download the main session from GCS; flaky network/GCS outages; container image builds that truncate staged files; custom containers missing staged artifacts.","solutions":["Let the worker restart/retry — this error intentionally forces re-provisioning; if persistent, check GCS staging bucket access and object integrity","Re-run the pipeline to re-stage the main session file","Verify network connectivity and GCS permissions from worker VMs","Check the staged files in the job's staging location; re-deploy if the session artifact is corrupt"],"exampleFix":"// before\n# empty session artifact staged; LoadMainSessionException raised\n// after\n# verify non-empty file in staging before job submission\ngsutil ls -l gs://<bucket>/staged/...session...\n# then re-run the pipeline to restage","handlingStrategy":"retry","validationCode":"if os.path.exists(session_file) and os.path.getsize(session_file) == 0:\n  print('empty session file, re-download required')","typeGuard":null,"tryCatchPattern":"try:\n  pickler.load_session(session_file)\nexcept LoadMainSessionException as e:\n  log.error('session download failed: %s', e)\n  os._exit(1)  # force worker restart so the file is re-fetched","preventionTips":["Monitor GCS staging bucket for truncated artifacts","Verify network reliability from worker VMs to GCS","Allow worker restarts/retries instead of suppressing this fatal error","Check custom container stages the main session correctly"],"tags":["apache-beam","dataflow","session-file","worker-startup"],"backgroundTag":"file-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}