{"record":{"id":"31fc78ee1f718dcb","repo":"Comfy-Org/ComfyUI","slug":"minimax-music3-text-encoder-checkpoint-is-missing","errorCode":null,"errorMessage":"MiniMax Music3 text encoder checkpoint is missing tokenizer_json","messagePattern":"MiniMax Music3 text encoder checkpoint is missing tokenizer_json","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"comfy/text_encoders/minimax_music.py","lineNumber":41,"sourceCode":"    \"decoder_intermediate_size\": 6144,\n    \"decoder_num_layers\": 4,\n}\n\n\ndef detect_merged_config(state_dict, prefix=\"\"):\n    return {\n        \"merged_qkv\": \"{}model.layers.0.self_attn.qkv_proj.weight\".format(prefix) in state_dict,\n        \"merged_mlp\": \"{}model.layers.0.mlp.gate_up_proj.weight\".format(prefix) in state_dict,\n        \"decoder_merged_qkv\": \"{}model.audio_decoder.layers.0.self_attn.qkv_proj.weight\".format(prefix) in state_dict,\n        \"decoder_merged_mlp\": \"{}model.audio_decoder.layers.0.mlp.gate_up_proj.weight\".format(prefix) in state_dict,\n    }\n\n\nclass MiniMaxMusic3Tokenizer:\n    def __init__(self, embedding_directory=None, tokenizer_data={}):\n        tokenizer_json = tokenizer_data.get(\"tokenizer_json\")\n        if tokenizer_json is None:\n            raise ValueError(\"MiniMax Music3 text encoder checkpoint is missing tokenizer_json\")\n        if torch.is_tensor(tokenizer_json):\n            tokenizer_json = tokenizer_json.detach().cpu().numpy().tobytes()\n        self.tokenizer_json = tokenizer_json\n        self.tokenizer = Tokenizer.from_str(tokenizer_json.decode(\"utf-8\"))\n        for token, expected in SPECIAL_TOKEN_IDS.items():\n            if self.tokenizer.token_to_id(token) != expected:\n                raise ValueError(f\"MiniMax Music3 tokenizer mismatch for {token}\")\n\n    def tokenize_with_weights(self, text, return_word_ids=False, **kwargs):\n        prompt = build_prompt(text, kwargs.get(\"lyrics\", \"\"))\n        token_ids = self.tokenizer.encode(prompt, add_special_tokens=False).ids\n        return {\n            \"minimax_music3\": [[(token, 1.0) for token in token_ids]],\n            \"seed\": int(kwargs.get(\"seed\", 0)),\n            \"max_audio_frames\": int(kwargs.get(\"max_audio_frames\", MAX_AUDIO_FRAMES)),\n            \"cfg_scale\": float(kwargs.get(\"cfg_scale\", CFG_SCALE)),\n            \"top_k\": int(kwargs.get(\"top_k\", CFG_TOP_K)),\n        }","sourceCodeStart":23,"sourceCodeEnd":59,"githubUrl":"https://github.com/Comfy-Org/ComfyUI/blob/1c6d8d45b3693bfbb32385b410d813a7fd6be216/comfy/text_encoders/minimax_music.py#L23-L59","documentation":"The MiniMax Music3 text encoder, like several ComfyUI bundled encoders, embeds its HuggingFace tokenizers JSON as a byte blob under 'tokenizer_json' in the checkpoint. MiniMaxMusic3Tokenizer.__init__ raises when tokenizer_data lacks that key, i.e. the checkpoint in models/text_encoders was not the ComfyUI-packaged bundle.","triggerScenarios":"Instantiating MiniMaxMusic3Tokenizer with tokenizer_data that has no 'tokenizer_json' entry — loading raw MiniMax Music3 HF weights converted by a generic converter instead of the ComfyUI bundle script.","commonSituations":"User downloaded the original HF repo weights directly instead of the ComfyUI-packaged safetensors; a conversion script filtered out non-float tensors, dropping the tokenizer blob.","solutions":["Download/use the ComfyUI-packaged MiniMax Music3 text encoder bundle that embeds tokenizer_json","Check the checkpoint: safetensors.torch.load_file(path).keys() must include 'tokenizer_json'; if missing, re-bundle","When re-bundling, store tokenizer.json bytes as a tensor under 'tokenizer_json' rather than shipping a side file"],"exampleFix":"// before\nsd = safetensors.torch.load_file(\"minimax_music3_te_raw.safetensors\")  # no tokenizer_json\n\n# after\n# bundle: torch.save({'tokenizer_json': torch.frombuffer(open('tokenizer.json','rb').read(), dtype=torch.uint8), ...})\nsd = safetensors.torch.load_file(\"minimax_music3_te_comfy.safetensors\")\nassert sd.get('tokenizer_json') is not None","handlingStrategy":"validation","validationCode":"sd = safetensors.torch.load_file(te_path)\nif sd.get('tokenizer_json') is None:\n    raise ValueError(f'{te_path} is not a ComfyUI-packaged Music3 encoder (no tokenizer_json)')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Download only the ComfyUI-packaged MiniMax Music3 encoder","Keep tokenizer and weights bundled together; never re-save with non-float tensors filtered out"],"tags":["text-encoder","tokenizer","checkpoint","minimax","music"],"backgroundTag":null,"analyzedSha":"1c6d8d45b3693bfbb32385b410d813a7fd6be216","analyzedAt":"2026-08-14T19:37:18.893Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}