{"record":{"id":"6af22dbb85ccc323","repo":"huggingface/candle","slug":"mergeable-ranks-is-not-an-object","errorCode":null,"errorMessage":"mergeable_ranks is not an object","messagePattern":"mergeable_ranks is not an object","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-transformers/src/models/metavoice.rs","lineNumber":220,"sourceCode":"                    None => candle::bail!(\"pat_str field is not a string\"),\n                    Some(pat_str) => fancy_regex::Regex::new(pat_str).map_err(E::wrap)?,\n                },\n            };\n            let offset = match json.get(\"offset\") {\n                None => candle::bail!(\"json object has no offset field\"),\n                Some(offset) => match offset.as_u64() {\n                    None => candle::bail!(\"offset field is not a positive int\"),\n                    Some(offset) => offset as usize,\n                },\n            };\n            let mut ranks = HashMap::new();\n            for id in 0u8..=255 {\n                ranks.insert(vec![id], id as u32);\n            }\n            let mergeable_ranks = match json.get(\"mergeable_ranks\") {\n                None => candle::bail!(\"json object has no mergeable_ranks field\"),\n                Some(mr) => match mr.as_object() {\n                    None => candle::bail!(\"mergeable_ranks is not an object\"),\n                    Some(mr) => mr,\n                },\n            };\n            for (key, value) in mergeable_ranks.iter() {\n                let value = match value.as_u64() {\n                    None => candle::bail!(\"mergeable_ranks '{key}' is not a u64\"),\n                    Some(value) => value as u32,\n                };\n                if value < 256 {\n                    continue;\n                }\n                // No escaping for other keys.\n                let key = key.as_bytes().to_vec();\n                ranks.insert(key, value);\n            }\n            Ok(Self {\n                re,\n                end_of_text,","sourceCodeStart":202,"sourceCodeEnd":238,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-transformers/src/models/metavoice.rs#L202-L238","documentation":"This error is thrown by Encoding::from_json in candle-transformers' metavoice model when the JSON tokenizer file's \"mergeable_ranks\" field exists but is not a JSON object (e.g. it is a string, array, number, or null). from_json expects a BPE-style mapping of byte sequences (hex keys) to u32 ranks, so it validates the field's type before iterating. It is a data-format validation failure, not a runtime bug.","triggerScenarios":"Calling Encoding::from_json (or the metavoice tokenizer loading path that wraps it) with a JSON file where \"mergeable_ranks\" is present but typed incorrectly — e.g. a serialized array of pairs, a JSON string containing the ranks, or null.","commonSituations":"Using a tokenizer.json from a different model family whose mergeable_ranks is stored in a different format; hand-editing or truncating the tokenizer JSON; downloading a corrupted or incomplete tokenizer file; using a JSON dump produced by a script that stringified the ranks.","solutions":["Inspect the tokenizer JSON and make \"mergeable_ranks\" a JSON object whose keys are hex byte strings and values are u64/u32 rank numbers","Regenerate or re-download the tokenizer file from the official MetaVoice model repository to match the expected format","Confirm you are passing the right tokenizer file (not a vocab.json or merges file from a different format) to from_json"],"exampleFix":"// before (invalid)\n{\"mergeable_ranks\": \"[{\"AA==\": 256}]\"}\n// after (valid)\n{\"mergeable_ranks\": {\"AA==\": 256, \"AQ==\": 257}}","handlingStrategy":"validation","validationCode":"let mr = json.get(\"mergeable_ranks\")\n    .ok_or(\"missing mergeable_ranks\")?;\nif !mr.is_object() {\n    return Err(\"mergeable_ranks must be a JSON object\");\n}","typeGuard":"fn is_object(v: &serde_json::Value) -> bool {\n    matches!(v, serde_json::Value::Object(_))\n}","tryCatchPattern":"let enc = Encoding::from_json(&json)\n    .map_err(|e| format!(\"bad tokenizer json: {e}\"))?;","preventionTips":["Validate tokenizer JSON structure (mergeable_ranks is an object of int ranks) before loading","Re-download tokenizer files from the official model repo rather than converting formats by hand","Add a JSON schema check in your pipeline for tokenizer files"],"tags":["rust","candle","json","tokenizer","schema-validation"],"backgroundTag":"schema-validation-failed","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}