keras-team/keras · error · ValueError

num_tokens must be set to use this layer. If the number of t

Error message

num_tokens must be set to use this layer. If the number of tokens is not known beforehand, use the IntegerLookup layer instead.

What it means

Error "num_tokens must be set to use this layer. If the number of tokens is not known beforehand, use the IntegerLookup layer instead." thrown in keras-team/keras.

Source

Thrown at keras/src/layers/preprocessing/category_encoding.py:102

            weight for each sample value when summing up in `count` mode.
            Not used in `"multi_hot"` or `"one_hot"` modes.
    """

    def __init__(
        self, num_tokens=None, output_mode="multi_hot", sparse=False, **kwargs
    ):
        super().__init__(**kwargs)

        # Support deprecated names for output_modes.
        if output_mode == "binary":
            output_mode = "multi_hot"

        # 'output_mode' must be one of ("count", "one_hot", "multi_hot")
        if output_mode not in ("count", "one_hot", "multi_hot"):
            raise ValueError(f"Unknown arg for output_mode: {output_mode}")

        if num_tokens is None:
            raise ValueError(
                "num_tokens must be set to use this layer. If the "
                "number of tokens is not known beforehand, use the "
                "IntegerLookup layer instead."
            )
        if num_tokens < 1:
            raise ValueError(
                f"`num_tokens` must be >= 1. Received: num_tokens={num_tokens}."
            )
        self.num_tokens = num_tokens
        self.output_mode = output_mode
        self.sparse = sparse
        self._allow_non_tensor_positional_args = True
        self._convert_input_args = False

    def _encode(self, inputs, count_weights=None):
        inputs = self.backend.core.convert_to_tensor(inputs)
        return numerical_utils.encode_categorical_inputs(
            inputs,

View on GitHub (pinned to 7a34a03db6)

When it happens

Trigger: Thrown at keras/src/layers/preprocessing/category_encoding.py:102 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of keras-team/keras@7a34a03db6 (2026-08-25). Data as JSON: /api/errors/00159bbed7bd94f4. Report an issue: GitHub.