vllm-project/vllm · error · ValueError

tok_pooling_type is not set; it should be resolved by ModelC

Error message

tok_pooling_type is not set; it should be resolved by ModelConfig before calling get_tok_pooling_type()

What it means

Tokenwise counterpart of get_seq_pooling_type: PoolerConfig.get_tok_pooling_type() raises when tok_pooling_type is None, meaning ModelConfig resolution (which decides between sequence and tokenwise pooling, e.g. for reward models) never populated it. Indicates the getter was called on an unresolved or sequence-only config.

Source

Thrown at vllm/config/pooler.py:172

                    "Resolved `pooling_type=%r` to `tok_pooling_type=%r`.",
                    pooling_type,
                    pooling_type,
                )
                self.tok_pooling_type = pooling_type  # type: ignore[assignment]
            else:
                raise NotImplementedError(pooling_type)

    def get_seq_pooling_type(self) -> SequencePoolingType:
        if self.seq_pooling_type is None:
            raise ValueError(
                "seq_pooling_type is not set; it should be resolved by"
                " ModelConfig before calling get_seq_pooling_type()"
            )
        return self.seq_pooling_type

    def get_tok_pooling_type(self) -> TokenPoolingType:
        if self.tok_pooling_type is None:
            raise ValueError(
                "tok_pooling_type is not set; it should be resolved by"
                " ModelConfig before calling get_tok_pooling_type()"
            )
        return self.tok_pooling_type

    def compute_hash(self) -> str:
        """
        WARNING: Whenever a new field is added to this config,
        ensure that it is included in the factors list if
        it affects the computation graph.

        Provide a hash that uniquely identifies all the configs
        that affect the structure of the computation
        graph from input ids/embeddings to the final hidden states,
        excluding anything before input ids/embeddings and after
        the final hidden states.
        """
        # no factors to consider.

View on GitHub (pinned to c794754062)

Solutions

Set tok_pooling_type explicitly ('ALL' or 'STEP') for direct construction.
Route through ModelConfig so resolution assigns it based on the model's task.
Check pooler_config.task / model type before assuming tokenwise pooling applies.

Example fix

# before
cfg = PoolerConfig()
t = cfg.get_tok_pooling_type()  # raises

# after
cfg = PoolerConfig(tok_pooling_type='ALL')
t = cfg.get_tok_pooling_type()

Defensive patterns

Strategy: validation

Validate before calling

def get_tok_type_or_default(cfg, default: str = "ALL") -> str:
    return cfg.tok_pooling_type if cfg.tok_pooling_type is not None else default

Type guard

def has_tok_pooling(cfg) -> bool:
    return cfg.tok_pooling_type is not None

Try / catch

try:
    ptype = cfg.get_tok_pooling_type()
except ValueError:
    ptype = "ALL"  # or ensure ModelConfig resolution ran first

Prevention

Construct PoolerConfig with explicit tok_pooling_type in unit tests that call the getter.
Confirm the model's task is token-level (e.g. reward/PRM) before requiring tok pooling.

When it happens

Trigger: Calling get_tok_pooling_type() on a PoolerConfig that was never run through ModelConfig resolution, or on a config for a sequence-pooling model where tok_pooling_type legitimately stays None.

Common situations: Token-classification/reward model code paths calling the getter before model config resolution; tests constructing PoolerConfig() directly.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/1d3a5b1844b34e61. Report an issue: GitHub.