rust-lang/rust · critical
alt layout should have a niche like the regular one
Error message
alt layout should have a niche like the regular one
What it means
Immediately after the end-biased alternative layout succeeds, the code assumes it contains a niche at least as usable as the head-biased one (`largest_niche` is `Some`). The premise is that the alternative run was only attempted because the regular layout *had* a niche and used the identical fields; end-biasing reorders but should not eliminate the niche. A missing niche here means the biasing changed which field exposes the niche — a bug in `univariant_biased`.
Source
Thrown at compiler/rustc_abi/src/layout.rs:303
// Don't try to calculate an end-biased layout for unsizable structs,
// otherwise we could end up with different layouts for
// Foo<Type> and Foo<dyn Trait> which would break unsizing.
if !matches!(kind, StructKind::MaybeUnsized) {
if let Some(niche) = layout.largest_niche {
let head_space = niche.offset.bytes();
let niche_len = niche.value.size(dl).bytes();
let tail_space = layout.size.bytes() - head_space - niche_len;
// This may end up doing redundant work if the niche is already in the last
// field (e.g. a trailing bool) and there is tail padding. But it's non-trivial
// to get the unpadded size so we try anyway.
if fields.len() > 1 && head_space != 0 && tail_space > 0 {
let alt_layout = self
.univariant_biased(fields, repr, kind, NicheBias::End)
.expect("alt layout should always work");
let alt_niche = alt_layout
.largest_niche
.expect("alt layout should have a niche like the regular one");
let alt_head_space = alt_niche.offset.bytes();
let alt_niche_len = alt_niche.value.size(dl).bytes();
let alt_tail_space =
alt_layout.size.bytes() - alt_head_space - alt_niche_len;
debug_assert_eq!(layout.size.bytes(), alt_layout.size.bytes());
let prefer_alt_layout =
alt_head_space > head_space && alt_head_space > tail_space;
debug!(
"sz: {}, default_niche_at: {}+{}, default_tail_space: {}, alt_niche_at/head_space: {}+{}, alt_tail: {}, num_fields: {}, better: {}\n\
layout: {}\n\
alt_layout: {}\n",
layout.size.bytes(),
head_space,
niche_len,
tail_space,View on GitHub (pinned to 22057b88b0)
Solutions
- Ensure `univariant_biased` populates `largest_niche` identically for both `NicheBias::Start` and `NicheBias::End` — the niche value depends on the field, not the bias.
- Reproduce with the offending struct and dump `largest_niche` for both layouts via the existing `debug!` call to find which field lost its niche under end-bias.
- Guard with a `debug_assert!` upstream so asymmetry is caught closer to the source.
- File an ICE with the struct definition if it reproduces on upstream rustc.
Example fix
// before — niche only set under Start bias
let largest_niche = if niche_bias == NicheBias::Start { find_niche() } else { None };
// after — niche is bias-independent
let largest_niche = find_niche(); Defensive patterns
Strategy: try-catch
Try / catch
// 'alt layout should have a niche like the regular one' is a .expect() asserting
// that the end-biased univariant pass produced a niche when the start-biased one
// did. A panic means the niche-finding logic is inconsistent; isolate and fall
// back to the start-biased layout (which the code already had).
let result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
calc.univariant(&fields, &repr, kind)
}));
match result {
Ok(Ok(layout)) => { /* primary (start-biased) layout is fine; use it */ }
Ok(Err(e)) => { /* surface layout error */ }
Err(payload) => {
log::error!(
"niche invariant violated on alt layout: {}",
payload.downcast_ref::<String>().map(|s| s.as_str())
.or_else(|| payload.downcast_ref::<&'static str>().copied())
.unwrap_or("?")
);
// The start-biased pass is the source of truth; if even that panicked,
// recompute with niche optimization disabled (repr without niche).
}
} Prevention
- Both [8] and [9] stem from the niche-biasing comparison inside univariant; they cannot be pre-validated by the caller, so defense = isolation + fallback, not input filtering.
- Keep the start-biased layout available as the authoritative fallback; the end-biased (alt) pass is only an optimization for niche placement near struct edges.
- When you hit this on a specific aggregate, record the field types and their niches — the bug is almost always a niche that the start pass found but the end pass dropped, so that field set is the minimization target.
When it happens
Trigger: `univariant`'s niche-optimization fallback branch calling `univariant_biased(..., NicheBias::End)`, getting `Ok(alt_layout)` where `alt_layout.largest_niche` is `None`, while the head-biased layout had `largest_niche = Some(..)`.
Common situations: A change to niche detection that depends on field order (so end-biasing reorders fields and loses the niche), a `repr`/`align` combination where the niche falls in padding only under one bias, or a refactor of `Niche` construction that forgot to populate `largest_niche` on the biased path.
Related errors
- alt layout should always work
- unsupported integer: {self:?}
- unsupported float: {self:?}
- `homogeneous_aggregate` should not be called for scalable ve
- aggregates can't have `FieldsShape::Primitive`
AI-assisted analysis of rust-lang/rust@22057b88b0 (2026-08-03).
Data as JSON: /data/errors/ec35c701de775a5b.json.
Report an issue: GitHub.