huggingface/candle · error

mergeable_ranks '{key}' is not a u64

Error message

mergeable_ranks '{key}' is not a u64

What it means

from_json iterates the mergeable_ranks object and requires every value to be a JSON integer (as_u64). This error is raised when a rank value in the "mergeable_ranks" map is not a number — e.g. a string, float, bool, nested object, or null. The library parses ranks as u64 and casts them to u32, so any non-integer value aborts loading.

Source

Thrown at candle-transformers/src/models/metavoice.rs:226

                Some(offset) => match offset.as_u64() {
                    None => candle::bail!("offset field is not a positive int"),
                    Some(offset) => offset as usize,
                },
            };
            let mut ranks = HashMap::new();
            for id in 0u8..=255 {
                ranks.insert(vec![id], id as u32);
            }
            let mergeable_ranks = match json.get("mergeable_ranks") {
                None => candle::bail!("json object has no mergeable_ranks field"),
                Some(mr) => match mr.as_object() {
                    None => candle::bail!("mergeable_ranks is not an object"),
                    Some(mr) => mr,
                },
            };
            for (key, value) in mergeable_ranks.iter() {
                let value = match value.as_u64() {
                    None => candle::bail!("mergeable_ranks '{key}' is not a u64"),
                    Some(value) => value as u32,
                };
                if value < 256 {
                    continue;
                }
                // No escaping for other keys.
                let key = key.as_bytes().to_vec();
                ranks.insert(key, value);
            }
            Ok(Self {
                re,
                end_of_text,
                offset,
                ranks,
                span: tracing::span!(tracing::Level::TRACE, "bpe"),
            })
        }

View on GitHub (pinned to d5fee525bf)

Solutions

1. Find the offending key named in the error message and change its value to a plain JSON integer
2. Regenerate the tokenizer JSON with the rank values emitted as numbers, not strings or floats
3. Re-download the original tokenizer file for the MetaVoice checkpoint instead of a converted copy

Example fix

// before
{"AA==": "256"}
// after
{"AA==": 256}

let mr = json["mergeable_ranks"].as_object()
    .ok_or("mergeable_ranks must be an object")?;
for (k, v) in mr {
    if v.as_u64().is_none() {
        return Err(format!("rank for '{k}' is not a u64"));
    }
}

fn all_u64(v: &serde_json::Value) -> bool {
    v.as_object().map_or(false, |o| {
        o.values().all(|x| x.is_u64())
    })
}

let enc = Encoding::from_json(&json)
    .map_err(|e| format!("invalid rank in tokenizer: {e}"))?;

When it happens

Trigger: Calling Encoding::from_json with a tokenizer JSON where at least one key in "mergeable_ranks" maps to a non-u64 value, such as "256" (string), 256.5 (float), or null.

Common situations: Tokenizer files converted between formats with quoted numeric values; hand-edited rank tables; scripts that dumped ranks as strings or floats; corrupted downloads where part of the JSON was mangled.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• mergeable_ranks is not an object
• unsupported type {:?} for '{name}' attribute in '{}' for {}
• {} is a dummy type and cannot be constructed
• {} is a dummy type and cannot be converted
• {} is a dummy type and cannot be converted to scalar

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/e0369c5452c4dadd. Report an issue: GitHub.