screenpipe/screenpipe · error · Error::Inference

rb split

Error message

rb split

What it means

Raised in `forward` when splitting the final `refpoint_embed` (cxcywh boxes) into two halves along the last axis via `ops::split_sections(.., &[2], -1)`. In mlx, a list argument is a set of split indices, so this expects the last dimension to be 4 (split at index 2 into [:, :2] and [:, 2:]). The library throws it when the reference-point tensor's last axis is not 4, making the split index invalid.

Source

Thrown at crates/screenpipe-rfdetr-mlx/src/decoder/mod.rs:678

        let qp0a = relu(&qp0)?;
        let query_pos = linear(&qp0a, &self.ref_point_head_w1, &self.ref_point_head_b1)?;

        // Decoder layer loop — uses RAW projector output (`tokens_flat`)
        // as memory, not enc_memory. No per-layer eval — let MLX
        // schedule the graph; the lazy-graph stack-overflow problem
        // we hit in the backbone (12 blocks deep) is much smaller here
        // (just 2 layers).
        for l in &self.layers {
            output = l.forward(&output, tokens_flat, &query_pos, &refpoints_input)?;
        }
        // Final LN.
        output = ln(&output, &self.final_norm_w, &self.final_norm_b)?;

        // Final heads.
        let logits = linear(&output, &self.class_head_w, &self.class_head_b)?;
        let bbox_delta = self.bbox_head.forward(&output)?;
        // Final bbox refinement vs. refpoint_embed (cxcywh).
        let rb_parts = ops::split_sections(&refpoint_embed, &[2], -1).map_err(err("rb split"))?;
        let bd_parts = ops::split_sections(&bbox_delta, &[2], -1).map_err(err("bd split"))?;
        let final_cxcy = bd_parts[0]
            .multiply(&rb_parts[1])
            .map_err(err("final cxcy mul"))?
            .add(&rb_parts[0])
            .map_err(err("final cxcy add"))?;
        let final_wh = exp_(&bd_parts[1])?
            .multiply(&rb_parts[1])
            .map_err(err("final wh mul"))?;
        let boxes =
            ops::concatenate_axis(&[&final_cxcy, &final_wh], -1).map_err(err("boxes cat"))?;

        Ok((boxes, logits))
    }

    /// Gather slices of `x` along axis 1 by integer indices `idx (B, K)`.
    /// Result: `(B, K, last_dim)`.
    fn gather(&self, x: &Array, idx: &Array, last_dim: i32) -> Result<Array> {

View on GitHub (pinned to 4ebf712990)

Solutions

1. Assert `refpoint_embed.shape()` is [B, 300, 4] before the split; fix the producer of the tensor if not.
2. Confirm the checkpoint's `refpoint_embed` weight has shape (300, 4); re-export weights for this model variant if it differs.
3. If using a cxcywh→xyxy conversion upstream, convert back to cxcywh (or adjust the split sections) before `forward`.
4. Inspect the wrapped mlx exception for the offending axis size.

Example fix

// before
let rb_parts = ops::split_sections(&refpoint_embed, &[2], -1).map_err(err("rb split"))?;
// after
assert_eq!(refpoint_embed.shape()[2], 4, "expected cxcywh (B, 300, 4)");
let rb_parts = ops::split_sections(&refpoint_embed, &[2], -1).map_err(err("rb split"))?;

// Before forward: refpoints must be cxcywh (B, 300, 4)
let s = refpoint_embed.shape();
if s.len() != 3 || s[2] != 4 {
    return Err(anyhow!("refpoint_embed must be (B, 300, 4), got {s:?}"));
}

fn is_cxcywh(a: &mlx_rs::Array) -> bool {
    a.shape().last().map_or(false, |&d| d == 4)
}

match decoder.forward(&enc, &refpoints) {
    Err(Error::Inference(m)) if m.contains("rb split") => {
        // convert xyxy -> cxcywh and retry once
        let cxcywh = xyxy_to_cxcywh(&refpoints)?;
        decoder.forward(&enc, &cxcywh)
    }
    other => other,
}

When it happens

Trigger: Final bbox refinement in `forward` where `refpoint_embed.shape()[-1] != 4` — e.g. a checkpoint whose learned refpoint embedding is (1, 300, 2) or (1, 300, 6), or a tensor that was previously reshaped/transposed incorrectly.

Common situations: Loading a DAB/LW-DETR checkpoint variant with a different box parameterization (e.g. xyxy or fewer dims); upstream code change altering refpoint layout; feeding proposal boxes instead of the learned embedding.

Related errors

• bbox split
• bd split
• final wh mul
• boxes cat
• at least 2 chunks

AI-assisted analysis of screenpipe/screenpipe@4ebf712990 (2026-09-01). Data as JSON: /api/errors/14d3c9eee7aab8ac. Report an issue: GitHub.