risingwavelabs/risingwave · error · SinkError::BigQuery

bigquery insert error: end of resp stream

Error message

bigquery insert error: end of resp stream

What it means

The Storage Write API AppendRows call must yield exactly one response per request. This error is returned when the response stream ended without yielding any response (None), meaning BigQuery closed or never produced a response for the append request.

Source

Thrown at src/connector/src/sink/big_query.rs:853

            .map_err(|e| SinkError::BigQuery(e.into()))?
        {
            Some(append_rows_response) => {
                if !append_rows_response.row_errors.is_empty() {
                    return Err(SinkError::BigQuery(anyhow::anyhow!(
                        "bigquery insert error {:?}",
                        append_rows_response.row_errors
                    )));
                }
                if let Some(google_cloud_googleapis::cloud::bigquery::storage::v1::append_rows_response::Response::Error(status)) = append_rows_response.response{
                            return Err(SinkError::BigQuery(anyhow::anyhow!(
                                "bigquery insert error {:?}",
                                status
                            )));
                        }
                yield ();
            }
            None => {
                return Err(SinkError::BigQuery(anyhow::anyhow!(
                    "bigquery insert error: end of resp stream",
                )));
            }
        }
    }
}

struct StorageWriterClient {
    #[expect(dead_code)]
    environment: Environment,
    request_sender: mpsc::UnboundedSender<AppendRowsRequest>,
}
impl StorageWriterClient {
    pub async fn new(
        credentials: CredentialsFile,
    ) -> Result<(Self, impl Stream<Item = Result<()>>)> {
        let ts_grpc = google_cloud_auth::token::DefaultTokenSourceProvider::new_with_credentials(
            Self::bigquery_grpc_auth_config(),

View on GitHub (pinned to 6469eb736d)

Solutions

1. Retry the sink; RisingWave fault-tolerance will replay the batch after recovery.
2. Check network stability between the RisingWave node and BigQuery (proxies, NAT idle timeouts); enable gRPC keepalive.
3. Reduce batch size / flush interval so individual AppendRows requests complete faster.
4. Upgrade RisingWave and the google-cloud-bigquery-storage client for stream-retry fixes; check BigQuery status for outages.

match sink_write_result {
    Err(SinkError::BigQuery(e)) if e.to_string().contains("end of resp stream") => {
        // transient gRPC stream close: rely on RisingWave recovery/replay or back off and retry
    }
    other => other?,
}

When it happens

Trigger: resp_to_stream awaiting the stream's .message() and receiving None before any append_rows_response - the gRPC response stream terminated prematurely after sending a batch.

Common situations: Network interruption or idle-timeout killing the bidirectional gRPC stream mid-batch; proxies/firewalls dropping long-lived streams; oversized batches taking too long to acknowledge; BigQuery service issues.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors

• failed to send request to {} {:?}
• end of stream
• bigquery insert error {:?}
• Cannot open client to compute node {addr:?}
• RpcError

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/3c43b12d78db9ccb. Report an issue: GitHub.