{"record":{"id":"d11b01cb1e7aa2da","repo":"hankcs/HanLP","slug":"coarse-tokenization-not-supported-for-language","errorCode":null,"errorMessage":"Coarse tokenization not supported for {language}. Please set language=\"zh\".","messagePattern":"Coarse tokenization not supported for (.+?)\\. Please set language=\"zh\"\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"plugins/hanlp_restful/hanlp_restful/__init__.py","lineNumber":326,"sourceCode":"            [['商品', '和', '服务', '。'],\n             ['当', '下雨天', '地面', '积水', '分', '外', '严重']]\n\n            # Multilingual tokenization by specifying language='mul':\n            HanLP.tokenize(\n                ['In 2021, HanLPv2.1 delivers state-of-the-art multilingual NLP techniques\n                 'to production environment.',\n                 '2021年、HanLPv2.1は次世代の最先端多言語NLP技術を本番環境に導入します。',\n                 '2021年 HanLPv2.1为生产环境带来次世代最先进的多语种NLP技术。'], language='mul')\n            [['In', '2021', ',', 'HanLPv2.1', 'delivers', 'state-of-the-art', 'multilingual',\n              'NLP', 'techniques', 'to', 'production', 'environment', '.'],\n             ['2021', '年', '、', 'HanLPv2.1', 'は', '次', '世代', 'の', '最', '先端', '多',\n              '言語', 'NLP', '技術', 'を', '本番', '環境', 'に', '導入', 'します', '。'],\n             ['2021', '年', 'HanLPv2.1', '为', '生产', '环境', '带来', '次世代', '最', '先进的',\n              '多', '语种', 'NLP', '技术', '。']]\n        \"\"\"\n        language = language or self._language\n        if coarse and language and language != 'zh':\n            raise NotImplementedError(f'Coarse tokenization not supported for {language}. Please set language=\"zh\".')\n        doc = self.parse(text=text, tasks='tok/coarse' if coarse is True else 'tok', language=language)\n        return next(iter(doc.values()))\n\n    def abstract_meaning_representation(self,\n                                        text: Union[str, List[str]] = None,\n                                        tokens: List[List[str]] = None,\n                                        language: str = None,\n                                        visualization: str = None,\n                                        ) -> List[Dict]:\n        \"\"\"Abstract Meaning Representation (AMR) captures “who is doing what to whom” in a sentence. Each sentence is\n        represented as a rooted, directed, acyclic graph consisting of nodes (concepts) and edges (relations).\n\n        Args:\n            text: A document (str), or a list of sentences (List[str]).\n            tokens: A list of sentences where each sentence is a list of tokens.\n            language: The language of input text or tokens. ``None`` to use the default language on server.\n            visualization: Set to `dot` or `svg` to obtain coresspodning visualization.\n","sourceCodeStart":308,"sourceCodeEnd":344,"githubUrl":"https://github.com/hankcs/HanLP/blob/ddb1299bddff079e447af52ec12549c50636bfa8/plugins/hanlp_restful/hanlp_restful/__init__.py#L308-L344","documentation":"The RESTful client's coarse tokenization (tok/coarse) is only implemented server-side for Chinese. Requesting coarse=True with any other language raises NotImplementedError.","triggerScenarios":"hanlp.tokenize(text, coarse=True) while the client was created with language='en'/'ja'/etc., or passing language != 'zh' explicitly with coarse=True.","commonSituations":"Applying the same coarse-tokenize pipeline used for Chinese to multilingual text; forgetting to reset language between requests.","solutions":["Use coarse=False for non-Chinese languages","Create a separate client with language='zh' for Chinese coarse tokenization needs","For non-Chinese sentence splitting, do it client-side before calling tokenize"],"exampleFix":"# before\nhanlp.tokenize('Hello world.', coarse=True)  # client language != zh\n# after\nhanlp.tokenize('Hello world.', coarse=False)","handlingStrategy":"validation","validationCode":"def can_coarse(client_language):\n    return client_language in (None, 'zh')","typeGuard":"def supports_coarse(language: str) -> bool:\n    return not language or language == 'zh'","tryCatchPattern":"try:\n    toks = hanlp.tokenize(text, coarse=True, language=lang)\nexcept NotImplementedError:\n    toks = hanlp.tokenize(text, coarse=False, language=lang)","preventionTips":["Check client._language before coarse=True","Keep a zh-specific client for coarse tokenization"],"tags":["tokenization","restful-api","language"],"backgroundTag":"unsupported-feature-for-language","analyzedSha":"ddb1299bddff079e447af52ec12549c50636bfa8","analyzedAt":"2026-08-27T03:36:54.287Z","schemaVersion":2},"datasetVersion":"2026-08-27T08:17:20.692Z"}