huggingface/transformers · error · ValueError

Worker process information is not available for seeding the

Error message

Worker process information is not available for seeding the generator. This may be because you are using multiprocessing without using a PyTorch DataLoader. The `seed` parameter can only be used when using multiprocessing with a PyTorch DataLoader. Please either use a single process or use a PyTorch DataLoader with multiple workers.

What it means

Raised by DataCollatorForLanguageModeling.create_rng when a seed was supplied but torch.utils.data.get_worker_info() returns None at call time. The per-worker deterministic seeding scheme (seed + worker_id) only works inside PyTorch DataLoader worker processes; outside them there is no worker id to derive a distinct stream, so the collator raises instead of silently producing collisions.

Source

Thrown at src/transformers/data/data_collator.py:761

        if mp.current_process().name == "MainProcess":
            # If we are in the main process, we create a generator object with the seed
            self.generator = self.get_generator(self.seed)
        else:
            # If we are in a worker process (i.e using multiprocessing), we need to set a unique seed for each
            # worker's generator, generated as the main seed + the worker's ID.
            # (https://pytorch.org/docs/stable/data.html#randomness-in-multi-process-data-loading)
            # Only PyTorch DataLoader allows us to access the worker ID, and so we check for this.
            import torch

            worker_info = torch.utils.data.get_worker_info()
            if worker_info is None:
                error_string = (
                    "Worker process information is not available for seeding the generator. This may be because",
                    "you are using multiprocessing without using a PyTorch DataLoader. The `seed` parameter can",
                    "only be used when using multiprocessing with a PyTorch DataLoader. Please either use a",
                    "single process or use a PyTorch DataLoader with multiple workers.",
                )
                raise ValueError(error_string)

            self.generator = self.get_generator(self.seed + worker_info.id)

    def torch_call(self, examples: list[list[int] | Any | dict[str, Any]]) -> dict[str, Any]:
        # Handle dict or lists with proper padding and conversion to tensor.

        if self.seed and self.generator is None:
            # If we have a seed, we need to create a generator object. Subsequent calls to this function will use the same generator.
            # If no seed supplied, we will use the global RNG
            self.create_rng()

        if isinstance(examples[0], Mapping):
            batch = pad_without_fast_tokenizer_warning(
                self.tokenizer, examples, return_tensors="pt", pad_to_multiple_of=self.pad_to_multiple_of
            )
        else:
            batch = {
                "input_ids": _torch_collate_batch(examples, self.tokenizer, pad_to_multiple_of=self.pad_to_multiple_of)

View on GitHub (pinned to a597f97485)

Solutions

Serve the collator through a PyTorch DataLoader with num_workers > 0 so worker info exists.
If single-process training is intended, drop the seed argument and seed the global RNG yourself (torch.manual_seed / np.random.seed).
If you need a seeded generator without DataLoader workers, subclass the collator and override create_rng to build a generator from self.seed alone.

Example fix

# before
collator = DataCollatorForLanguageModeling(tokenizer=tok, mlm=True, seed=42)
batch = collator(samples)  # raises: no worker info in main process

# after (single process: use global RNG seeding)
torch.manual_seed(42)
collator = DataCollatorForLanguageModeling(tokenizer=tok, mlm=True)
batch = collator(samples)

# after (deterministic per-worker seeding)
loader = DataLoader(ds, batch_size=8, num_workers=4, collate_fn=collator)

Defensive patterns

Strategy: validation

Validate before calling

import torch

def make_collator(tokenizer, seed, use_loader_workers):
    if seed is not None and not use_loader_workers:
        # no worker info will exist; seed globally instead
        torch.manual_seed(seed)
        return DataCollatorForLanguageModeling(tokenizer=tokenizer)
    return DataCollatorForLanguageModeling(tokenizer=tokenizer, seed=seed)

Try / catch

try:
    batch = collator(features)
except ValueError as e:
    if 'Worker process information' in str(e):
        # fall back to global RNG / single-process determinism
        torch.manual_seed(42)
        batch = collator_unseeded(features)
    else:
        raise

Prevention

Only pass seed when the collator is used as a DataLoader collate_fn with num_workers > 0.
In single-process training, rely on torch.manual_seed / np.random.seed for reproducibility.
Unit-test the collator without the seed argument to avoid depending on worker info.

When it happens

Trigger: Passing seed=... to DataCollatorForLanguageModeling and then calling the collator (directly or via a plain multiprocessing pool, a HF Trainer without a PyTorch DataLoader worker context, or a manual training loop) where get_worker_info() is None.

Common situations: Using the collator with num_workers=0 in a DataLoader (main-process collation), a custom multiprocessing.DataLoader replacement, or unit tests that call collator(batch) directly; upgrading transformers versions where seed support was newly added.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/6b138a30850a0329. Report an issue: GitHub.