What causes "cannot export more than U32_MAX files" in rust-lang/rust?

Reduce the number of distinct source files the crate feeds to the compiler (deduplicate generated files, avoid per-iteration `include!`/`concat!` patterns that mint new SourceFiles).

rust-lang/rust · critical

cannot export more than U32_MAX files

Error message

cannot export more than U32_MAX files

What it means

Thrown in rustc's rmeta encoder while serializing a local SourceFile's span: the per-crate metadata index assigned to a SourceFile must fit in a u32 (encoder.rs:330 narrows a usize via try_into().expect()). The compiler allocates one index per distinct SourceFile in the crate's SourceMap, and u32::MAX (~4.29 billion) is the hard ceiling the on-disk format can address. Hitting it means the bookkeeping produced more source files than the format can represent.

Source

Thrown at compiler/rustc_metadata/src/rmeta/encoder.rs:330

            //
            // All of this logic ensures that the final result of deserialization is a 'normal'
            // Span that can be used without any additional trouble.
            let metadata_index = {
                // Introduce a new scope so that we drop the 'read()' temporary
                match &*source_file.external_src.read() {
                    ExternalSource::Foreign { metadata_index, .. } => *metadata_index,
                    src => panic!("Unexpected external source {src:?}"),
                }
            };

            (SpanKind::Foreign, metadata_index)
        } else {
            // Record the fact that we need to encode the data for this `SourceFile`
            let source_files =
                s.required_source_files.as_mut().expect("Already encoded SourceMap!");
            let (metadata_index, _) = source_files.insert_full(source_file_index);
            let metadata_index: u32 =
                metadata_index.try_into().expect("cannot export more than U32_MAX files");

            (SpanKind::Local, metadata_index)
        };

        // Encode the start position relative to the file start, so we profit more from the
        // variable-length integer encoding.
        let lo = self.lo - source_file.start_pos;

        // Encode length which is usually less than span.hi and profits more
        // from the variable-length integer encoding that we use.
        let len = self.hi - self.lo;

        let tag = SpanTag::new(kind, ctxt, len.0 as usize);
        tag.encode(s);
        if tag.context().is_none() {
            ctxt.encode(s);
        }
        lo.encode(s);

View on GitHub (pinned to 22057b88b0)

Solutions

Reduce the number of distinct source files the crate feeds to the compiler (deduplicate generated files, avoid per-iteration `include!`/`concat!` patterns that mint new SourceFiles).
If the count is genuinely that high, split the crate so each crate stays well under u32::MAX files.
Confirm it is not a double-registration bug by checking whether the panic reproduces after `cargo clean`; if counts look sane in a small repro, file a rustc issue with the repro.
As a last resort on a debug build, verify SourceMap growth is not being driven by an internal leak.

Defensive patterns

Strategy: validation

Validate before calling

fn exported_file_count_within_u32(count: usize) -> bool {
    count <= u32::MAX as usize
}
// before invoking any bulk-export/encode path:
assert!(
    exported_file_count_within_u32(crate_files.len()),
    "rustc_metadata cannot encode more than u32::MAX files; got {}",
    crate_files.len()
);

Type guard

// rmeta encodes file indices as u32; guard every external index.
fn is_valid_rmeta_file_index(i: usize) -> bool {
    i < u32::MAX as usize
}

Prevention

Cap the number of source modules / generated files in a single crate well below u32::MAX (in practice below ~4 billion; real projects hit unrelated limits first, but codegen/script generators that emit millions of files can approach this).
Split large generated crates into multiple smaller crates rather than one mega-crate.
If you drive rustc programmatically via the metadata encoder API, pre-aggregate and count the file set before calling the encoder and abort with a clear message rather than letting rustc panic.
Never feed synthesized/compiled metadata whose file table you did not construct yourself.

When it happens

Trigger: A single crate records more than u32::MAX distinct SourceFile entries during `encode_span` / source-map population — e.g. pathological macro/codegen that mints a new SourceFile per iteration, `include!`/`include_str!` driven loops, or a proc-macro that synthesizes vast numbers of named spans. Practically only reproducible with generated builds far beyond normal scale.

Common situations: Not hit by normal projects. Seen (if ever) with extreme code generation, build-script-driven mass file emission, or as a symptom of a compiler bug double-counting SourceFiles. A clean rebuild typically reveals it is not a steady-state limit a real codebase reaches.

Related errors

AI-assisted analysis of rust-lang/rust@22057b88b0 (2026-08-03). Data as JSON: /data/errors/75f2e0bf13f977b5.json. Report an issue: GitHub.