huggingface/candle · error

INTERNAL ERROR inconsistent lhs and rhs {lhs:?} {rhs:?}

Error message

INTERNAL ERROR inconsistent lhs and rhs {lhs:?} {rhs:?}

What it means

In a streaming module's step(), the left and right StreamTensor inputs must both be present or both be absent; a mismatch (one Some, one None) indicates the streaming pipeline desynchronized, which candle treats as an internal invariant violation rather than a user-facing error.

Source

Thrown at candle-core/src/streaming.rs:191

            BinOp::Div => Tensor::div(lhs, rhs),
        }
    }

    pub fn step(&mut self, lhs: &StreamTensor, rhs: &StreamTensor) -> Result<StreamTensor> {
        let lhs = StreamTensor::cat2(&self.prev_lhs, lhs, self.dim)?;
        let rhs = StreamTensor::cat2(&self.prev_rhs, rhs, self.dim)?;
        let lhs_len = lhs.seq_len(self.dim)?;
        let rhs_len = rhs.seq_len(self.dim)?;
        let common_len = usize::min(lhs_len, rhs_len);
        let (lhs, prev_lhs) = lhs.split(self.dim, common_len)?;
        let (rhs, prev_rhs) = rhs.split(self.dim, common_len)?;
        let ys = match (lhs.0, rhs.0) {
            (Some(lhs), Some(rhs)) => {
                let ys = self.forward(&lhs, &rhs)?;
                StreamTensor::from_tensor(ys)
            }
            (None, None) => StreamTensor::empty(),
            (lhs, rhs) => crate::bail!("INTERNAL ERROR inconsistent lhs and rhs {lhs:?} {rhs:?}"),
        };
        self.prev_lhs = prev_lhs;
        self.prev_rhs = prev_rhs;
        Ok(ys)
    }
}

/// Simple wrapper that doesn't do any buffering.
pub struct Map<T: crate::Module>(T);

impl<T: crate::Module> StreamingModule for Map<T> {
    fn reset_state(&mut self) {}

    fn step(&mut self, xs: &StreamTensor) -> Result<StreamTensor> {
        xs.apply(&self.0)
    }
}

View on GitHub (pinned to d5fee525bf)

Solutions

1. Audit your streaming loop to ensure both input streams are fed the same number of steps and finish at the same time.
2. Check StreamTensor is_empty() on both inputs before calling step and handle end-of-stream on either side jointly.
3. Verify that upstream modules output matching stream lengths; pad/trim streams to align them.

Example fix

// before
ys = module.step(&lhs, &rhs)?; // may be one empty, one Some
// after
if lhs.is_empty() != rhs.is_empty() {
    // align or end both streams before stepping
    return Err(anyhow!("streams desynchronized"));
}
ys = module.step(&lhs, &rhs)?;

if lhs.is_empty() != rhs.is_empty() {
    return Err(anyhow!("stream inputs desynchronized: lhs empty={} rhs empty={}", lhs.is_empty(), rhs.is_empty()));
}
let ys = module.step(&lhs, &rhs)?;

match module.step(&lhs, &rhs) {
    Ok(ys) => ys,
    Err(e) if e.to_string().contains("inconsistent lhs and rhs") => {
        // re-align streams: drop/pad frames until both sides agree, then retry
        ...
    }
    Err(e) => return Err(e.into()),
}

When it happens

Trigger: Calling StreamTensor::step on a streaming module where one input stream produced a value while the other produced empty() at the same step — e.g. feeding streams of unequal lengths or misaligned step counts.

Common situations: Streaming a sequence through a module with two inputs (e.g. attention with separate query/key streams) where one input ran out of frames earlier than the other; off-by-one in a streaming loop.

Related errors

• GroupNorm doesn't support causal evaluation.

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/1f546e0faf7ecaae. Report an issue: GitHub.