apache/beam · critical · LoadMainSessionException

Session file found, but empty

Error message

Session file found, but empty: %s.%s

What it means

During worker startup, create_harness loads the pickled main session file containing the user's pipeline dependencies. If the session file exists but has zero bytes, the worker cannot restore user code; when running under dill (is_currently_dill) this is treated as fatal via LoadMainSessionException ('Session file found, but empty'), crashing the worker to force a restart and re-download.

Solutions

  1. Let the worker restart/retry — this error intentionally forces re-provisioning; if persistent, check GCS staging bucket access and object integrity
  2. Re-run the pipeline to re-stage the main session file
  3. Verify network connectivity and GCS permissions from worker VMs
  4. Check the staged files in the job's staging location; re-deploy if the session artifact is corrupt

Example fix

// before
# empty session artifact staged; LoadMainSessionException raised
// after
# verify non-empty file in staging before job submission
gsutil ls -l gs://<bucket>/staged/...session...
# then re-run the pipeline to restage
Defensive patterns

Strategy: retry

Validate before calling

if os.path.exists(session_file) and os.path.getsize(session_file) == 0:
  print('empty session file, re-download required')

Try / catch

try:
  pickler.load_session(session_file)
except LoadMainSessionException as e:
  log.error('session download failed: %s', e)
  os._exit(1)  # force worker restart so the file is re-fetched

Prevention

When it happens

Trigger: The staged session file (~templated or staged to the worker) downloads as 0 bytes — transient fetch failure — while the worker uses the dill pickler.

Common situations: Dataflow workers failing to download the main session from GCS; flaky network/GCS outages; container image builds that truncate staged files; custom containers missing staged artifacts.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/1f5c29514f710b26. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/runners/worker/sdk_worker_main.py:412

    warn_msg = ' Functions defined in __main__ (interactive session) may fail.'
    err_msg = ' Functions defined in __main__ (interactive session) will ' \
      'almost certainly fail.'
  elif pickler.is_currently_cloudpickle():
    warn_msg = ' User registered objects (e.g. schema, logical type) through' \
        'registeries may not be effective'
    err_msg = ''
  if semi_persistent_directory:
    session_file = os.path.join(
        semi_persistent_directory, 'staged', names.PICKLED_MAIN_SESSION_FILE)
    if os.path.isfile(session_file):
      # If the expected session file is present but empty, it's likely that
      # the user code run by this worker will likely crash at runtime.
      # This can happen if the worker fails to download the main session.
      # Raise a fatal error and crash this worker, forcing a restart.
      if os.path.getsize(session_file) == 0:
        if pickler.is_currently_dill():
          # Potenitally transient error, unclear if still happening.
          raise LoadMainSessionException(
              'Session file found, but empty: %s.%s' % (session_file, err_msg))
        else:
          _LOGGER.warning('Empty session file: %s.%s', warn_msg, session_file)
      else:
        pickler.load_session(session_file)
    else:
      _LOGGER.warning('No session file found: %s.%s', warn_msg, session_file)
  else:
    _LOGGER.warning('No semi_persistent_directory found: %s', warn_msg)


if __name__ == '__main__':
  main(sys.argv)

View on GitHub (pinned to 12126d8942)