risingwavelabs/risingwave · error

failed to send the rebuild-sink request to the reader

Error message

failed to send the rebuild-sink request to the reader

What it means

During a barrier with a vnode bitmap update, the sink executor sends a RebuildSink request over a oneshot-backed channel to the sink reader task, which owns sink rebuilding. If the reader task has already terminated (channel closed), send fails and the executor converts it into this error, meaning the executor can no longer coordinate state restoration with the reader.

Source

Thrown at src/stream/src/executor/sink.rs:532

                                is_checkpoint: barrier.kind.is_checkpoint(),
                                new_vnode_bitmap: update_vnode_bitmap.clone(),
                                is_stop: barrier.is_stop(actor_id),
                                schema_change,
                                wait_log_store_flush,
                            },
                        )
                        .await?;

                    let mutation = barrier.mutation.clone();
                    yield Message::Barrier(barrier);
                    if F::REBUILD_SINK_ON_UPDATE_VNODE_BITMAP
                        && let Some(new_vnode_bitmap) = update_vnode_bitmap.clone()
                    {
                        let (tx, rx) = oneshot::channel();
                        rebuild_sink_tx
                            .send(RebuildSinkMessage::RebuildSink(new_vnode_bitmap, tx))
                            .map_err(|_| {
                                anyhow!("failed to send the rebuild-sink request to the reader")
                            })?;
                        rx.await
                            .map_err(|_| anyhow!("failed to wait for sink rebuild to finish"))?;
                    }
                    post_flush.post_yield_barrier().await?;

                    if let Some(mutation) = mutation.as_deref() {
                        match mutation {
                            Mutation::Pause => {
                                log_writer.pause()?;
                                is_paused = true;
                            }
                            Mutation::Resume => {
                                log_writer.resume()?;
                                is_paused = false;
                            }
                            Mutation::Throttle(fragment_to_apply) => {
                                if let Some(entry) = fragment_to_apply.get(&fragment_id)

View on GitHub (pinned to 6469eb736d)

Solutions

1. Check sink reader task logs (look for panics or early exits in the sink consumer) preceding this error
2. Restart the actor/recovery — this usually resolves after the affected actor is rebuilt
3. If reproducible, inspect lifecycle of rebuild_sink_rx in execute_consume_log to see why the reader dropped early
4. Report to RisingWave with actor/sink id if it recurs; it indicates a sink executor lifecycle bug

match rebuild_sink_tx.send(RebuildSinkMessage::RebuildSink(bitmap, tx)) {
    Ok(()) => {}
    Err(_) => return Err(anyhow!("failed to send the rebuild-sink request to the reader")),
}

When it happens

Trigger: rebuild_sink_tx.send(RebuildSinkMessage::RebuildSink(...)) returns Err because the receiving half held by the sink reader/consumer task was dropped — i.e. the reader exited or panicked before the barrier arrived.

Common situations: Sink reader task panicked or was cancelled during recovery, actor being migrated/killed mid-barrier, or a bug in sink lifecycle management causing premature drop of the reader.

Related errors

• failed to receive rebuild sink notify
• channel closed
• Lance fragment write task stopped before accepting a record
• unable to send response: {:?}
• (dynamic: channel send error on shutdown signal)

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/777e0c1f4d163c75. Report an issue: GitHub.