pydantic/monty · critical

StringId overflow while building reverse interns map

Error message

StringId overflow while building reverse interns map

What it means

`build_string_id_by_name` rebuilds the name→`StringId` reverse map and re-derives each id from the index plus `INTERN_STRING_ID_OFFSET`; the `expect` panics if that sum exceeds the ID's backing integer. Same invariant as the intern-time `StringId overflow` check, evaluated when the lookup table is constructed.

Source

Thrown at crates/monty/src/intern.rs:1578

        }
    }
}

/// Builds the `String → StringId` reverse map from the `strings` vector.
///
/// Used both at fresh [`Interns::new`] time and after deserialization. The
/// ids start at [`INTERN_STRING_ID_OFFSET`] because slots `< OFFSET` are
/// reserved for ASCII single-character strings and the [`StaticStrings`]
/// table — those are handled by the cheap branches at the top of
/// [`Interns::get_string_id_by_name`] and never enter this map.
fn build_string_id_by_name(strings: &[WithHash<String>]) -> AHashMap<String, StringId> {
    strings
        .iter()
        .enumerate()
        .map(|(index, entry)| {
            let id = StringId(
                u32::try_from(INTERN_STRING_ID_OFFSET + index)
                    .expect("StringId overflow while building reverse interns map"),
            );
            (entry.value().clone(), id)
        })
        .collect()
}

impl Interns {
    /// Builds the runtime table from a finished parse/prepare interner and the
    /// functions compiled against it.
    pub fn new(interner: InternerBuilder, functions: Vec<Function>) -> Self {
        // `InternerBuilder` already maintains the `String → StringId` map
        // during the parse/prepare phase to deduplicate `intern` calls;
        // we move it across so `Interns::get_string_id_by_name` doesn't
        // have to rebuild the same table from `strings`.
        Self {
            strings: interner.strings,
            bytes: interner.bytes,
            long_ints: interner.long_ints,

View on GitHub (pinned to adc986b362)

Solutions

1. Bound the intern table size per compilation/execution and reuse tables.
2. Widen `StringId` if the workload needs a larger ID space.
3. Report as a bug if reached with realistic input.

Example fix

// before
.expect("StringId overflow while building reverse interns map")
// after
u32::try_from(INTERN_STRING_ID_OFFSET + index)
    .map_err(|_| CompileError::TooManyInternedStrings)?

assert!(strings.len() + INTERN_STRING_ID_OFFSET < u32::MAX as usize, "StringId space exhausted");

When it happens

Trigger: Calling `InternTable::from` (which builds the reverse map) on a table containing more strings than fit in `StringId` after the offset.

Common situations: Billions of interned strings in one table, or a table-reuse bug allowing unbounded growth before the reverse map is built.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• LongIntId overflow
• StringId overflow
• Invalid static string ID
• dict_keys view must reference a dict
• dict_items view must always reference a dict

AI-assisted analysis of pydantic/monty@adc986b362 (2026-09-13). Data as JSON: /api/errors/764b75c312d96d82. Report an issue: GitHub.