rust-lang/rust · critical

alt layout should have a niche like the regular one

Error message

alt layout should have a niche like the regular one

What it means

Immediately after the end-biased alternative layout succeeds, the code assumes it contains a niche at least as usable as the head-biased one (`largest_niche` is `Some`). The premise is that the alternative run was only attempted because the regular layout *had* a niche and used the identical fields; end-biasing reorders but should not eliminate the niche. A missing niche here means the biasing changed which field exposes the niche — a bug in `univariant_biased`.

Source

Thrown at compiler/rustc_abi/src/layout.rs:303

            // Don't try to calculate an end-biased layout for unsizable structs,
            // otherwise we could end up with different layouts for
            // Foo<Type> and Foo<dyn Trait> which would break unsizing.
            if !matches!(kind, StructKind::MaybeUnsized) {
                if let Some(niche) = layout.largest_niche {
                    let head_space = niche.offset.bytes();
                    let niche_len = niche.value.size(dl).bytes();
                    let tail_space = layout.size.bytes() - head_space - niche_len;

                    // This may end up doing redundant work if the niche is already in the last
                    // field (e.g. a trailing bool) and there is tail padding. But it's non-trivial
                    // to get the unpadded size so we try anyway.
                    if fields.len() > 1 && head_space != 0 && tail_space > 0 {
                        let alt_layout = self
                            .univariant_biased(fields, repr, kind, NicheBias::End)
                            .expect("alt layout should always work");
                        let alt_niche = alt_layout
                            .largest_niche
                            .expect("alt layout should have a niche like the regular one");
                        let alt_head_space = alt_niche.offset.bytes();
                        let alt_niche_len = alt_niche.value.size(dl).bytes();
                        let alt_tail_space =
                            alt_layout.size.bytes() - alt_head_space - alt_niche_len;

                        debug_assert_eq!(layout.size.bytes(), alt_layout.size.bytes());

                        let prefer_alt_layout =
                            alt_head_space > head_space && alt_head_space > tail_space;

                        debug!(
                            "sz: {}, default_niche_at: {}+{}, default_tail_space: {}, alt_niche_at/head_space: {}+{}, alt_tail: {}, num_fields: {}, better: {}\n\
                            layout: {}\n\
                            alt_layout: {}\n",
                            layout.size.bytes(),
                            head_space,
                            niche_len,
                            tail_space,

View on GitHub (pinned to 22057b88b0)

Solutions

  1. Ensure `univariant_biased` populates `largest_niche` identically for both `NicheBias::Start` and `NicheBias::End` — the niche value depends on the field, not the bias.
  2. Reproduce with the offending struct and dump `largest_niche` for both layouts via the existing `debug!` call to find which field lost its niche under end-bias.
  3. Guard with a `debug_assert!` upstream so asymmetry is caught closer to the source.
  4. File an ICE with the struct definition if it reproduces on upstream rustc.

Example fix

// before — niche only set under Start bias
let largest_niche = if niche_bias == NicheBias::Start { find_niche() } else { None };

// after — niche is bias-independent
let largest_niche = find_niche();
Defensive patterns

Strategy: try-catch

Try / catch

// 'alt layout should have a niche like the regular one' is a .expect() asserting
// that the end-biased univariant pass produced a niche when the start-biased one
// did. A panic means the niche-finding logic is inconsistent; isolate and fall
// back to the start-biased layout (which the code already had).
let result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
    calc.univariant(&fields, &repr, kind)
}));
match result {
    Ok(Ok(layout)) => { /* primary (start-biased) layout is fine; use it */ }
    Ok(Err(e))     => { /* surface layout error */ }
    Err(payload)   => {
        log::error!(
            "niche invariant violated on alt layout: {}",
            payload.downcast_ref::<String>().map(|s| s.as_str())
                .or_else(|| payload.downcast_ref::<&'static str>().copied())
                .unwrap_or("?")
        );
        // The start-biased pass is the source of truth; if even that panicked,
        // recompute with niche optimization disabled (repr without niche).
    }
}

Prevention

When it happens

Trigger: `univariant`'s niche-optimization fallback branch calling `univariant_biased(..., NicheBias::End)`, getting `Ok(alt_layout)` where `alt_layout.largest_niche` is `None`, while the head-biased layout had `largest_niche = Some(..)`.

Common situations: A change to niche detection that depends on field order (so end-biasing reorders fields and loses the niche), a `repr`/`align` combination where the niche falls in padding only under one bias, or a refactor of `Niche` construction that forgot to populate `largest_niche` on the biased path.

Related errors


AI-assisted analysis of rust-lang/rust@22057b88b0 (2026-08-03). Data as JSON: /data/errors/ec35c701de775a5b.json. Report an issue: GitHub.