risingwavelabs/risingwave · error · anyhow::Error
end of log stream
Error message
end of log stream
What it means
The in-memory log store's reader attempted to fetch the next log item but its buffer was exhausted (None from the queue), so it reports the log stream has ended. Reading past the end of an in-memory log stream is treated as a fatal error rather than a normal EOF.
Source
Thrown at src/stream/src/common/log_store_impl/in_mem.rs:226
};
} else {
self.epoch_progress = Consuming(next_epoch);
}
self.latest_offset = TruncateOffset::Barrier {
epoch: current_epoch,
};
Ok((
current_epoch,
LogStoreReadItem::Barrier {
is_checkpoint: options.is_checkpoint,
new_vnode_bitmap: options.new_vnode_bitmap,
is_stop: options.is_stop,
schema_change: options.schema_change,
},
))
}
},
None => Err(anyhow!("end of log stream")),
},
AwaitingTruncate { .. } => std::future::pending().await,
}
}
fn truncate(&mut self, offset: TruncateOffset) -> LogStoreResult<()> {
// check the truncate offset is higher than prev truncate offset
if self.truncate_offset >= offset {
return Err(anyhow!(
"truncate offset {:?} but prev truncate offset is {:?}",
offset,
self.truncate_offset
));
}
// check the truncate offset does not exceed the latest possible offset
if offset > self.latest_offset {
return Err(anyhow!(View on GitHub (pinned to 6469eb736d)
Solutions
- Check why the producer stopped appending (upstream actor failure) and restart the source
- Verify barrier/stream recovery logic does not read past latest_offset
- Re-run/recover the streaming job so the log store is rebuilt with correct offsets
Defensive patterns
Strategy: retry
Try / catch
match log_reader.next_item().await {
Err(e) if e.to_string().contains("end of log stream") => {
// wait for producer or trigger recovery/restart of the actor
tokio::time::sleep(Duration::from_millis(100)).await;
}
other => other?,
} Prevention
- Ensure producers append before consumers advance
- Add liveness monitoring for stream producer actors
- Model recovery so readers never read past latest_offset
When it happens
Trigger: next_item called after all appended items were consumed and the stream was not extended; occurs when a stream consumer outpaces producers or after the log was truncated/reset.
Common situations: Actor/barrier reader advancing beyond the latest appended entry, a crashed or stalled producer leaving the stream short, or stream recovery logic reading a closed in-memory log.
Related errors
- truncate offset {:?} but prev truncate offset is {:?}
- truncate at {:?} but latest offset is {:?}
- unable to send sealed epoch
- should not call rewind on it
- Filter can only receive bool array
AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11).
Data as JSON: /api/errors/7aac65433178037e.
Report an issue: GitHub.