{"record":{"id":"0c09b5450a5dc463","repo":"openai/whisper","slug":"unsupported-language-language","errorCode":null,"errorMessage":"Unsupported language: {language}","messagePattern":"Unsupported language: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"whisper/tokenizer.py","lineNumber":380,"sourceCode":"        special_tokens=special_tokens,\n    )\n\n\n@lru_cache(maxsize=None)\ndef get_tokenizer(\n    multilingual: bool,\n    *,\n    num_languages: int = 99,\n    language: Optional[str] = None,\n    task: Optional[str] = None,  # Literal[\"transcribe\", \"translate\", None]\n) -> Tokenizer:\n    if language is not None:\n        language = language.lower()\n        if language not in LANGUAGES:\n            if language in TO_LANGUAGE_CODE:\n                language = TO_LANGUAGE_CODE[language]\n            else:\n                raise ValueError(f\"Unsupported language: {language}\")\n\n    if multilingual:\n        encoding_name = \"multilingual\"\n        language = language or \"en\"\n        task = task or \"transcribe\"\n    else:\n        encoding_name = \"gpt2\"\n        language = None\n        task = None\n\n    encoding = get_encoding(name=encoding_name, num_languages=num_languages)\n\n    return Tokenizer(\n        encoding=encoding, num_languages=num_languages, language=language, task=task\n    )\n","sourceCodeStart":362,"sourceCodeEnd":396,"githubUrl":"https://github.com/openai/whisper/blob/5f86d1d86363843179951550570367b37c5d6f78/whisper/tokenizer.py#L362-L396","documentation":"get_tokenizer() accepts a language only if it is either a key of LANGUAGES (a code like 'en', 'de') or a key of TO_LANGUAGE_CODE (an English name like 'english', 'german', which it then converts). Anything else — before the multilingual branch even runs — raises ValueError('Unsupported language: ...').","triggerScenarios":"get_tokenizer(True, language='zh-CN') (locale form instead of 'zh'); language='eng' (ISO-639-2/b code); passing None-with-typo or a name with trailing whitespace ('english ') so neither dict matches; also raised via transcribe(model, path, language=<bad>).","commonSituations":"Feeding BCP-47 locale tags from web requests directly into language=; using 3-letter ISO codes from other datasets; uppercase already handled (.lower() is applied) but hyphens/whitespace are not.","solutions":["Normalize to the base two-letter code: language.split('-')[0].strip().lower() and verify against whisper.tokenizer.LANGUAGES","Pass the English display name ('german') only if spelled exactly as in TO_LANGUAGE_CODE","Validate user-supplied languages against whisper.tokenizer.LANGUAGES before calling transcribe()"],"exampleFix":"# before\nresult = whisper.transcribe(model, path, language=\"zh-CN\")  # ValueError\n\n# after\nfrom whisper.tokenizer import LANGUAGES\nlang = \"zh-CN\".split(\"-\")[0].strip().lower()\nassert lang in LANGUAGES, f\"unsupported: {lang}\"\nresult = whisper.transcribe(model, path, language=lang)","handlingStrategy":"validation","validationCode":"from whisper.tokenizer import LANGUAGES, TO_LANGUAGE_CODE\n\ndef normalize_language(raw: str) -> str:\n    lang = raw.strip().lower()\n    if lang in LANGUAGES:\n        return lang\n    if lang in TO_LANGUAGE_CODE:\n        return TO_LANGUAGE_CODE[lang]\n    base = lang.split(\"-\")[0].split(\"_\")[0]\n    if base in LANGUAGES:\n        return base\n    raise ValueError(f\"Unsupported language: {raw}\")","typeGuard":"def is_supported_language(v: str) -> bool:\n    v = v.strip().lower()\n    return v in LANGUAGES or v in TO_LANGUAGE_CODE","tryCatchPattern":null,"preventionTips":["Normalize BCP-47 tags to the primary subtag before passing language to whisper","Expose a whitelist of supported languages at your API boundary and reject others with a 4xx"],"tags":["tokenizer","language","validation","configuration"],"backgroundTag":null,"analyzedSha":"5f86d1d86363843179951550570367b37c5d6f78","analyzedAt":"2026-08-14T18:53:59.547Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}