janhq/jan · error · io::Error

Array length {} is unreasonably large

Error message

Array length {} is unreasonably large

What it means

Returned by `read_gguf_value` when an `Array` value's element count (u64 LE) exceeds 1,000,000. Arrays in GGUF metadata (e.g. tokenizer merges, vocabulary) can be large but should not exceed this cap; the guard prevents allocating and iterating an attacker-controlled huge count.

Source

Thrown at src-tauri/plugins/tauri-plugin-llamacpp/src/gguf/helpers.rs:99

        GgufValueType::Uint8 => Ok(reader.read_u8()?.to_string()),
        GgufValueType::Int8 => Ok(reader.read_i8()?.to_string()),
        GgufValueType::Uint16 => Ok(reader.read_u16::<LittleEndian>()?.to_string()),
        GgufValueType::Int16 => Ok(reader.read_i16::<LittleEndian>()?.to_string()),
        GgufValueType::Uint32 => Ok(reader.read_u32::<LittleEndian>()?.to_string()),
        GgufValueType::Int32 => Ok(reader.read_i32::<LittleEndian>()?.to_string()),
        GgufValueType::Float32 => Ok(reader.read_f32::<LittleEndian>()?.to_string()),
        GgufValueType::Bool => Ok((reader.read_u8()? != 0).to_string()),
        GgufValueType::String => read_gguf_string(reader),
        GgufValueType::Uint64 => Ok(reader.read_u64::<LittleEndian>()?.to_string()),
        GgufValueType::Int64 => Ok(reader.read_i64::<LittleEndian>()?.to_string()),
        GgufValueType::Float64 => Ok(reader.read_f64::<LittleEndian>()?.to_string()),
        GgufValueType::Array => {
            let elem_type_u32 = reader.read_u32::<LittleEndian>()?;
            let elem_type = GgufValueType::try_from(elem_type_u32)?;
            let len = reader.read_u64::<LittleEndian>()?;

            if len > 1_000_000 {
                return Err(io::Error::new(
                    io::ErrorKind::InvalidData,
                    format!("Array length {} is unreasonably large", len),
                ));
            }

            if len > 24 {
                skip_array_data(reader, elem_type, len)?;
                return Ok(format!(
                    "<Array of type {:?} with {} elements, data skipped>",
                    elem_type, len
                ));
            }

            let mut elems = Vec::with_capacity(len as usize);
            for _ in 0..len {
                elems.push(read_gguf_value(reader, elem_type)?);
            }
            Ok(format!("[{}]", elems.join(", ")))

View on GitHub (pinned to fad3f12a14)

Solutions

1. Audit `skip_array_data` against the GGUF spec for the offending element type to confirm the skip uses the correct byte width.
2. Hex-dump the region around the failure to see whether the length looks plausible.
3. If the array is legitimately large (a big tokenizer), raise the cap to a value that comfortably covers real files (e.g. 10,000,000).
4. Verify checksum and re-download if corrupted.

Example fix

// before
if len > 1_000_000 {
    return Err(io::Error::new(InvalidData, format!("Array length {} is unreasonably large", len)));
}

// after - cap with context
const MAX_ARRAY_LEN: u64 = 10_000_000;
if len > MAX_ARRAY_LEN {
    return Err(io::Error::new(InvalidData,
        format!("array length {} exceeds cap {} (suspect misalignment)", len, MAX_ARRAY_LEN)));
}

const MAX_ARRAY: u64 = 1_000_000;
fn safe_array_len(reader: &mut impl ReadBytesExt) -> Option<u64> {
    reader.read_u64::<LittleEndian>().ok().filter(|&n| n <= MAX_ARRAY)
}

null

match read_gguf_value(reader, value_type) {
    Ok(v) => Ok(v),
    Err(e) if e.to_string().contains("Array length") => {
        Err(CorruptFile("oversized array length".into()))
    }
    Err(e) => Err(e.into()),
}

When it happens

Trigger: Decoding an `Array` value whose length field decodes above 1,000,000. The most common cause is stream misalignment: the element-type u32 or the length u64 was read from the wrong offset because a prior field had the wrong width. A genuinely huge array (very large tokenizer) could also trip it but is uncommon.

Common situations: Misalignment after skipping a previous array whose element stride was wrong; a corrupted file; a malformed GGUF produced by a buggy converter. Note that arrays of length > 24 are already skipped (`skip_array_data`), so the cap is specifically about refusing pathological counts before the skip loop.

Related errors

• String length {} is unreasonably large
• Not a GGUF file
• <Utf8Error>
• Unknown GGUF value type: {}
• Error reading metadata entry {}: {}

AI-assisted analysis of janhq/jan@fad3f12a14 (2026-08-12). Data as JSON: /api/errors/8c045b7251056d93. Report an issue: GitHub.