hankcs/HanLP · error · NotImplementedError

Coarse tokenization not supported for {language}. Please set

Error message

Coarse tokenization not supported for {language}. Please set language="zh".

What it means

The RESTful client's coarse tokenization (tok/coarse) is only implemented server-side for Chinese. Requesting coarse=True with any other language raises NotImplementedError.

Source

Thrown at plugins/hanlp_restful/hanlp_restful/__init__.py:326

            [['商品', '和', '服务', '。'],
             ['当', '下雨天', '地面', '积水', '分', '外', '严重']]

            # Multilingual tokenization by specifying language='mul':
            HanLP.tokenize(
                ['In 2021, HanLPv2.1 delivers state-of-the-art multilingual NLP techniques
                 'to production environment.',
                 '2021年、HanLPv2.1は次世代の最先端多言語NLP技術を本番環境に導入します。',
                 '2021年 HanLPv2.1为生产环境带来次世代最先进的多语种NLP技术。'], language='mul')
            [['In', '2021', ',', 'HanLPv2.1', 'delivers', 'state-of-the-art', 'multilingual',
              'NLP', 'techniques', 'to', 'production', 'environment', '.'],
             ['2021', '年', '、', 'HanLPv2.1', 'は', '次', '世代', 'の', '最', '先端', '多',
              '言語', 'NLP', '技術', 'を', '本番', '環境', 'に', '導入', 'します', '。'],
             ['2021', '年', 'HanLPv2.1', '为', '生产', '环境', '带来', '次世代', '最', '先进的',
              '多', '语种', 'NLP', '技术', '。']]
        """
        language = language or self._language
        if coarse and language and language != 'zh':
            raise NotImplementedError(f'Coarse tokenization not supported for {language}. Please set language="zh".')
        doc = self.parse(text=text, tasks='tok/coarse' if coarse is True else 'tok', language=language)
        return next(iter(doc.values()))

    def abstract_meaning_representation(self,
                                        text: Union[str, List[str]] = None,
                                        tokens: List[List[str]] = None,
                                        language: str = None,
                                        visualization: str = None,
                                        ) -> List[Dict]:
        """Abstract Meaning Representation (AMR) captures “who is doing what to whom” in a sentence. Each sentence is
        represented as a rooted, directed, acyclic graph consisting of nodes (concepts) and edges (relations).

        Args:
            text: A document (str), or a list of sentences (List[str]).
            tokens: A list of sentences where each sentence is a list of tokens.
            language: The language of input text or tokens. ``None`` to use the default language on server.
            visualization: Set to `dot` or `svg` to obtain coresspodning visualization.

View on GitHub (pinned to ddb1299bdd)

Solutions

1. Use coarse=False for non-Chinese languages
2. Create a separate client with language='zh' for Chinese coarse tokenization needs
3. For non-Chinese sentence splitting, do it client-side before calling tokenize

Example fix

# before
hanlp.tokenize('Hello world.', coarse=True)  # client language != zh
# after
hanlp.tokenize('Hello world.', coarse=False)

def can_coarse(client_language):
    return client_language in (None, 'zh')

def supports_coarse(language: str) -> bool:
    return not language or language == 'zh'

try:
    toks = hanlp.tokenize(text, coarse=True, language=lang)
except NotImplementedError:
    toks = hanlp.tokenize(text, coarse=False, language=lang)

When it happens

Trigger: hanlp.tokenize(text, coarse=True) while the client was created with language='en'/'ja'/etc., or passing language != 'zh' explicitly with coarse=True.

Common situations: Applying the same coarse-tokenize pipeline used for Chinese to multilingual text; forgetting to reset language between requests.

Related errors

• Unsupported tagging scheme {tagging_scheme}.

AI-assisted analysis of hankcs/HanLP@ddb1299bdd (2026-08-27). Data as JSON: /api/errors/d11b01cb1e7aa2da. Report an issue: GitHub.