{"record":{"id":"695dc2e5c5800b45","repo":"grpc/grpc-go","slug":"failed-to-send-client-headers-to-external-processo","errorCode":null,"errorMessage":"failed to send client headers to external processor server: %v","messagePattern":"failed to send client headers to external processor server: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/xds/httpfilter/extproc/ext_proc.go","lineNumber":468,"sourceCode":"\t\tonFinishFunc := func(error) {\n\t\t\ttime.AfterFunc(ocs.config.deferredCloseTimeout, ocs.procCancel)\n\t\t}\n\t\tnewOpts := append(opts, grpc.OnFinish(onFinishFunc))\n\n\t\tif ocs.dataplaneStream, err = newStream(ocs.ctx, newOpts...); err != nil {\n\t\t\tocs.cancel()\n\t\t\treturn nil, ocs.streamError(err)\n\t\t}\n\t\tocs.recordMetric(clientHeadersDurationMetric, timeSince(ocs.clientHeadersStartTime).Seconds())\n\t\t// Start background goroutine to receive any messages from the external\n\t\t// processor server and discard them.\n\t\tgo ocs.discardProcessorResponsesLoop()\n\n\t\t// If the request header processing mode is set to \"Send\", forward the\n\t\t// headers to the external processor server.\n\t\tif i.config.processingModes.requestHeaderMode == modeSend {\n\t\t\tif err = ocs.sendToProcessor(ocs.requestHeaders(outgoingMD, added)); err != nil {\n\t\t\t\treturn ocs.handleInitError(fmt.Errorf(\"failed to send client headers to external processor server: %v\", err), newStream, opts...)\n\t\t\t}\n\t\t}\n\n\t\treturn ocs, nil\n\t}\n\n\t// Normal mode.\n\tcs := &clientStream{\n\t\tcommonStream:             csCommon,\n\t\tprocStreamFailed:         grpcsync.NewEvent(),\n\t\tprocStreamBypass:         grpcsync.NewEvent(),\n\t\tmutatedReqBuffer:         buffer.NewUnbounded[*v3procservicepb.StreamedBodyResponse](),\n\t\tmutatedRespBuffer:        buffer.NewUnbounded[*v3procservicepb.StreamedBodyResponse](),\n\t\tresponseHeadersReady:     grpcsync.NewEvent(),\n\t\tresponseTrailerReady:     grpcsync.NewEvent(),\n\t\tdataplaneSetup:           make(chan struct{}),\n\t\tprocSendCh:               make(chan *v3procservicepb.ProcessingRequest),\n\t\trequestForwardLoopDoneCh: make(chan struct{}),","sourceCodeStart":450,"sourceCodeEnd":486,"githubUrl":"https://github.com/grpc/grpc-go/blob/0c51461d27177d997e14c642fe18c11668fc09a3/internal/xds/httpfilter/extproc/ext_proc.go#L450-L486","documentation":"Returned in observability mode (ext_proc.go:468) when sending the request headers ProcessingRequest to the ext_proc server fails. It is wrapped via handleInitError, so in deny mode the RPC fails with codes.Internal and in allow mode it bypasses ext_proc.","triggerScenarios":"Triggered when requestHeaderMode is SEND (observability mode, ext_proc.go:466) and ocs.sendToProcessor(ocs.requestHeaders(...)) returns a non-nil error — e.g. the proc stream died, write after close, context deadline, or transport reset right after stream creation.","commonSituations":"Ext_proc server closes the stream immediately after accept (misconfigured handler), context deadline too short for the configured timeout, connection RST during header send, or server-side panic aborting the stream.","solutions":["Enable failure_mode_allow so a header-send failure degrades to direct dataplane instead of failing the RPC.","Inspect ext_proc server logs for the immediate close/panic and fix the handler that rejects the first message.","Increase server.Timeout (grpc_service timeout) or remove it if the headers round-trip exceeds it.","Verify observability_mode is actually intended — if not, disable it to switch to the normal (synchronous) path."],"exampleFix":"// before: header send failure in observability mode fails the RPC\nfilter.observability_mode = true\nfilter.failure_mode_allow = false\n\n// after: tolerate send failures; dataplane proceeds without ext_proc\nfilter.observability_mode = true\nfilter.failure_mode_allow = true","handlingStrategy":"try-catch","validationCode":"// Nothing to validate pre-call; the failure happens on send. Confirm the\n// observability mode + timeout are sane to reduce the chance.\nfunc saneObservabilityConfig(f baseConfig) error {\n    if f.observabilityMode && f.server.Timeout != 0 && f.server.Timeout < 100*time.Millisecond {\n        return fmt.Errorf(\"observability header send likely to exceed server.Timeout=%v\", f.server.Timeout)\n    }\n    return nil\n}","typeGuard":null,"tryCatchPattern":"err := conn.Invoke(ctx, method, req, resp)\nif st, ok := status.FromError(err); ok && st.Code() == codes.Internal &&\n    strings.Contains(st.Message(), \"failed to send client headers to external processor\") {\n    // header send failed in observability mode: enable failure_mode_allow to bypass\n}","preventionTips":["Enable failure_mode_allow so observability-mode send failures do not break the dataplane RPC.","Confirm the ext_proc server does not immediately close the stream after accept.","Keep server.Timeout generous enough for the headers round-trip (or unset it).","Only enable observability_mode when you genuinely do not need synchronous processing."],"tags":["grpc","xds","extproc","envoy","observability-mode","headers","failure-mode-allow"],"backgroundTag":null,"analyzedSha":"0c51461d27177d997e14c642fe18c11668fc09a3","analyzedAt":"2026-08-11T14:49:15.055Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}