hankcs/HanLP · error · ValueError

Unsupported normalization method {normalize}

Error message

Unsupported normalization method {normalize}

What it means

After selecting pretrained vectors, index_word2vec_with_vocab can normalize them; supported schemes are 'norm' (divide by row L2 norm), 'l2' (F.normalize p=2), and 'std' (divide by global std). Any other normalize string raises, so typos or unsupported schemes fail fast rather than silently skipping normalization.

Source

Thrown at hanlp/layers/embeddings/util.py:74

    if unk_id_offset:
        unk_embeds = torch.zeros(unk_id_offset, pret_matrix.size(1))
        if init and init != 'zeros':
            if init == 'uniform':
                init = embedding_uniform
            else:
                raise ValueError(f'Unsupported init {init}')
            unk_embeds = init(unk_embeds)
        pret_matrix = torch.cat([pret_matrix, unk_embeds])
    ids = torch.LongTensor(ids)
    embedding = pret_matrix.index_select(0, ids)
    if normalize == 'norm':
        embedding /= (torch.norm(embedding, dim=1, keepdim=True) + 1e-12)
    elif normalize == 'l2':
        embedding = torch.nn.functional.normalize(embedding, p=2, dim=1)
    elif normalize == 'std':
        embedding /= torch.std(embedding)
    else:
        raise ValueError(f'Unsupported normalization method {normalize}')
    return embedding


def build_word2vec_with_vocab(embed: Union[str, int],
                              vocab: Vocab,
                              extend_vocab=True,
                              unk=None,
                              lowercase=False,
                              trainable=False,
                              init='zeros',
                              normalize=None) -> nn.Embedding:
    """Build word2vec embedding and a vocab.

    Args:
        embed:
        vocab: The vocabulary from training set.
        extend_vocab: Unlock vocabulary of training set to add those tokens in pretrained embedding file.
        unk: UNK token.

View on GitHub (pinned to ddb1299bdd)

Solutions

1. Use one of 'norm', 'l2', 'std'
2. Pass normalize=None (or omit) to skip normalization

Example fix

# before
emb = build_word2vec_with_vocab(p, vocab, normalize='minmax')  # error
# after
emb = build_word2vec_with_vocab(p, vocab, normalize='l2')

assert normalize in (None, 'norm', 'l2', 'std'), f'bad normalize: {normalize}'

def valid_normalize(norm) -> bool:
    return norm in (None, 'norm', 'l2', 'std')

When it happens

Trigger: Passing normalize='none', normalize=None is fine but 'max', 'minmax', 'unit' etc. raise the error.

Common situations: Assuming other normalization names from sklearn/gensim apply; typo in config keys.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• Unsupported init {init}
• error
• Unsupported argument type: {item}
• Unrecognized mapper type {mapper}
• self.model.config.pad_token_id has to be defined.

AI-assisted analysis of hankcs/HanLP@ddb1299bdd (2026-08-27). Data as JSON: /api/errors/16bf8d892047a637. Report an issue: GitHub.