hankcs/HanLP · error · NotImplementedError

Not supported.

Error message

Not supported.

What it means

Word2VecDataset.load_file raises NotImplementedError because word2vec embedding components consume in-memory token lists (built in build_dataloader), not corpus files. The class only exists to carry transforms over tokens; reading training samples from a path is unsupported.

Source

Thrown at hanlp/layers/embeddings/word2vec.py:172

                padding.append(vocab.pad_idx)
            word_dropout = WordDropout(self.word_dropout, vocab.unk_idx, exclude_tokens=padding)
        else:
            word_dropout = None
        return Word2VecEmbeddingModule(self.field, embed, word_dropout=word_dropout, cpu=self.cpu,
                                       second_channel=self.second_channel, num_tokens_in_trn=num_tokens_in_trn,
                                       unk_idx=vocab.unk_idx)

    def transform(self, vocabs: VocabDict = None, **kwargs) -> Optional[Callable]:
        assert vocabs is not None
        if self.field not in vocabs:
            vocabs[self.field] = Vocab(pad_token=self.pad, unk_token=self.unk)
        return super().transform(**kwargs)


class Word2VecDataset(TransformableDataset):

    def load_file(self, filepath: str):
        raise NotImplementedError('Not supported.')


class Word2VecEmbeddingComponent(TorchComponent):

    def __init__(self, **kwargs) -> None:
        """ Toy example of Word2VecEmbedding. It simply returns the embedding of a given word

        Args:
            **kwargs:
        """
        super().__init__(**kwargs)
        self._tokenizer: Trie = None

    def build_dataloader(self, data: List[str], shuffle=False, device=None, logger: logging.Logger = None,
                         doc2vec=False, batch_size=32, **kwargs) -> DataLoader:
        dataset = Word2VecDataset([{'token': x} for x in data], transform=self._tokenize if doc2vec else self.vocabs)
        return PadSequenceDataLoader(dataset, device=device, batch_size=batch_size)

View on GitHub (pinned to ddb1299bdd)

Solutions

1. Pass in-memory data: Word2VecDataset([{'token': x} for x in data], transform=...)
2. Train word2vec externally (gensim/fastText) and load vectors via the embedding config

Example fix

# before
ds = Word2VecDataset(['corpus.txt'])  # error
# after
ds = Word2VecDataset([{'token': sent} for sent in sents], transform=vocabs)

assert isinstance(data, list), 'pass in-memory token lists to Word2VecDataset'

def is_token_list(data) -> bool:
    return isinstance(data, list) and not isinstance(data, str)

When it happens

Trigger: Constructing Word2VecDataset with a file path (forcing load_file) or calling load_file directly.

Common situations: Attempting to train word2vec vectors from a text corpus inside HanLP; reusing the dataset class generically.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• error
• Not supported.
• output ({}) must be of type bool or str
• Call fit or load before evaluate.
• Unrecognized devices {devices}

AI-assisted analysis of hankcs/HanLP@ddb1299bdd (2026-08-27). Data as JSON: /api/errors/2af6ebbed3f86f13. Report an issue: GitHub.