grpc/grpc-go · error

failed to send client headers to external processor server:

Error message

failed to send client headers to external processor server: %v

What it means

Raised in observability mode when sendToProcessor(requestHeaders) fails on the freshly opened proc stream. The stream was established but the very first message (client request headers) could not be sent, and the error is routed through handleInitError.

Source

Thrown at internal/xds/httpfilter/extproc/ext_proc.go:334

	// they go over the wire, we must establish the ext_proc stream now rather
	// than deferring it.
	var err error
	if csCommon.procStream, err = i.procClient.Process(procCtx); err != nil {
		return csCommon.handleInitError(fmt.Errorf("failed to create a stream to external processor: %v", err), newStream, opts...)
	}

	// Observability mode.
	if i.config.observabilityMode {
		ocs := &observabilityClientStream{
			commonStream: csCommon,
			procRecvDone: make(chan struct{}),
		}

		// If the request header processing mode is set to "Send", forward the
		// headers to the external processor server.
		if i.config.processingModes.requestHeaderMode == modeSend {
			if err = ocs.sendToProcessor(ocs.requestHeaders(outgoingMD, added)); err != nil {
				return ocs.handleInitError(fmt.Errorf("failed to send client headers to external processor server: %v", err), newStream, opts...)
			}
		}

		// Defer the closing of ext proc stream by the defined defferred close
		// timeout to allow the server to read all messages from the proc stream.
		onFinishFunc := func(error) {
			time.AfterFunc(ocs.config.deferredCloseTimeout, ocs.procCancel)
		}
		newOpts := append(opts, grpc.OnFinish(onFinishFunc))

		if ocs.dataplaneStream, err = newStream(ocs.ctx, newOpts...); err != nil {
			ocs.cancel()
			return nil, err
		}
		ocs.recordMetric(clientHeadersDurationMetric, timeSince(ocs.clientHeadersStartTime).Seconds())
		// Start background goroutine to receive any messages from the external
		// processor server and discard them.
		go ocs.discardProcessorResponsesLoop()

View on GitHub (pinned to 03255a9237)

Solutions

1. Check server logs for why it tore down the stream right after Process() opened.
2. Ensure the external processor supports observability mode and the ProcessingRequest schema version the client sends.
3. Enable failure_mode_allow so handleInitError falls through to the dataplane stream instead of failing the user RPC.
4. If request-header sending is not actually needed, set request_header_mode to SKIP for the observability path.

Example fix

// before
observability_mode: true
request_header_mode: SEND   # server RSTs on first send
failure_mode_allow: false

// after
observability_mode: true
request_header_mode: SEND
failure_mode_allow: true

// Observability-mode send failures also go through handleInitError and respect
// failure_mode_allow. Enable it to bypass rather than fail the user RPC.

When it happens

Trigger: Observability mode is on, requestHeaderMode == modeSend, and procStream.Send of the requestHeaders ProcessingRequest fails — e.g. the server RST_STREAM'd immediately, transport went away, or context was canceled before the send completed.

Common situations: Server closes the proc stream as soon as it opens (misconfigured observability mode, version skew on the ProcessingRequest schema). Network reset after the initial handshake. procCtx canceled because the parent RPC context was canceled.

Related errors

• failed to create a stream to external processor: %v
• last resolver error: %v
• last connection error: %v
• last connection error: %v; last resolver error: %v
• all SubConns are in TransientFailure, last connection error:

AI-assisted analysis of grpc/grpc-go@03255a9237 (2026-08-07). Data as JSON: /api/errors/695dc2e5c5800b45. Report an issue: GitHub.