{"record":{"id":"fde60e71d7f1c06c","repo":"apache/beam","slug":"job-v-failed-w","errorCode":null,"errorMessage":"job %v failed:\n%w","messagePattern":"job (.+?) failed:\n%w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"sdks/go/pkg/beam/runners/universal/runnerlib/job.go","lineNumber":159,"sourceCode":"\n\t\t\tvar b strings.Builder\n\t\t\tif resp.GetTime() != \"\" {\n\t\t\t\tfmt.Fprintf(&b, \"(time=%v)\", resp.GetTime())\n\t\t\t}\n\t\t\tif resp.GetMessageId() != \"\" {\n\t\t\t\tfmt.Fprintf(&b, \"(id=%v)\", resp.GetMessageId())\n\t\t\t}\n\t\t\tb.WriteString(resp.GetMessageText())\n\t\t\ttext := b.String()\n\n\t\t\tlog.Output(ctx, messageSeverity(resp.GetImportance()), 1, text)\n\n\t\t\tif resp.GetImportance() >= jobpb.JobMessage_JOB_MESSAGE_ERROR {\n\t\t\t\terrReceived = true\n\t\t\t\tmostRecentError = resp.GetMessageText()\n\n\t\t\t\tif jobFailed {\n\t\t\t\t\treturn errors.Errorf(\"job %v failed:\\n%w\", jobID, errors.New(mostRecentError))\n\t\t\t\t}\n\t\t\t}\n\n\t\tdefault:\n\t\t\treturn errors.Errorf(\"unexpected job update: %v\", msg.String())\n\t\t}\n\t}\n}\n\nfunc messageSeverity(importance jobpb.JobMessage_MessageImportance) log.Severity {\n\tswitch importance {\n\tcase jobpb.JobMessage_JOB_MESSAGE_ERROR:\n\t\treturn log.SevError\n\tcase jobpb.JobMessage_JOB_MESSAGE_WARNING:\n\t\treturn log.SevWarn\n\tcase jobpb.JobMessage_JOB_MESSAGE_BASIC:\n\t\treturn log.SevInfo\n\tcase jobpb.JobMessage_JOB_MESSAGE_DEBUG, jobpb.JobMessage_JOB_MESSAGE_DETAILED:","sourceCodeStart":141,"sourceCodeEnd":177,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/go/pkg/beam/runners/universal/runnerlib/job.go#L141-L177","documentation":"The universal runner's WaitForCompletion streams JobMessages from the harness. When a message with severity >= JOB_MESSAGE_ERROR arrives and the job has also reached a FAILED state, it wraps the most recent error text as 'job %v failed:\\n%w'. This is the standard surfaced error for a remote pipeline that died on the runner side.","triggerScenarios":"Running a pipeline through the universal runner (Dataflow, Flink via portable runner) where the job transitions to FAILED and an error job message was received over the Lens/LegacyJobService stream.","commonSituations":"Worker crashes (OOM, missing dependencies in the worker container), user code panics in DoFns, runner-side resource exhaustion; the inner text carries the harness's actual error message.","solutions":["Read the wrapped message after 'failed:' — it contains the actual harness error and fix that root cause","Check runner-side logs (worker logs) for stack traces around the failure time","Re-run with a local/direct runner first to reproduce user-code errors locally","Increase worker resources or container memory if the failure is OOM-related"],"exampleFix":"// before\nif jobFailed {\n\treturn errors.Errorf(\"job %v failed:\\n%w\", jobID, errors.New(mostRecentError))\n}\n// after\nif jobFailed {\n\treturn errors.Errorf(\"job %v failed:\\n%s\", jobID, mostRecentError) // inspect mostRecentError for root cause\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"if err := runner.WaitForCompletion(ctx, conn, jobID, deadline, printer); err != nil {\n\tif strings.HasPrefix(err.Error(), \"job \") && strings.Contains(err.Error(), \"failed:\") {\n\t\t// parse text after 'failed:' for the harness root cause and log it\n\t}\n\treturn err\n}","preventionTips":["Always read the wrapped inner message — it carries the real harness error","Reproduce pipelines locally with the direct runner before remote submission","Set worker container memory/CPU generously to avoid OOM failures","Keep SDK and runner versions aligned"],"tags":["go","beam","universal-runner","remote-execution"],"backgroundTag":"upstream-api-error","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}