{"record":{"id":"dc207f93c2a5e9e2","repo":"apache/beam","slug":"job-s-failed","errorCode":null,"errorMessage":"Job %s failed","messagePattern":"Job (.+?) failed","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"sdks/go/pkg/beam/runners/dataflow/dataflowlib/job.go","lineNumber":305,"sourceCode":"// Errors are always terminal.\nfunc currentStateMessage(currentState, jobID string) (bool, string, error) {\n\tswitch currentState {\n\t// Add all Terminal Success stats here.\n\tcase \"JOB_STATE_DONE\", \"JOB_STATE_CANCELLED\", \"JOB_STATE_DRAINED\", \"JOB_STATE_UPDATED\":\n\t\tvar state string\n\t\tswitch currentState {\n\t\tcase \"JOB_STATE_DONE\":\n\t\t\tstate = \"succeeded!\"\n\t\tcase \"JOB_STATE_CANCELLED\":\n\t\t\tstate = \"cancelled\"\n\t\tcase \"JOB_STATE_DRAINED\":\n\t\t\tstate = \"drained\"\n\t\tcase \"JOB_STATE_UPDATED\":\n\t\t\tstate = \"updated\"\n\t\t}\n\t\treturn true, fmt.Sprintf(\"Job %v %v\", jobID, state), nil\n\tcase \"JOB_STATE_FAILED\":\n\t\treturn true, \"\", errors.Errorf(\"Job %s failed\", jobID)\n\tcase \"JOB_STATE_RUNNING\":\n\t\treturn false, \"Job still running ...\", nil\n\tdefault:\n\t\treturn false, fmt.Sprintf(\"Job state: %v ...\", currentState), nil\n\t}\n}\n\n// NewClient creates a new dataflow client with default application credentials\n// and CloudPlatformScope. The Dataflow endpoint is optionally overridden.\nfunc NewClient(ctx context.Context, endpoint string) (*df.Service, error) {\n\tcl, err := google.DefaultClient(ctx, df.CloudPlatformScope)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tclient, err := df.New(cl)\n\tif err != nil {\n\t\treturn nil, err\n\t}","sourceCodeStart":287,"sourceCodeEnd":323,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/go/pkg/beam/runners/dataflow/dataflowlib/job.go#L287-L323","documentation":"When the Dataflow service reports the job state JOB_STATE_FAILED, currentStateMessage (called from WaitForCompletion) returns a terminal error 'Job <jobID> failed'. This is not an SDK malfunction: the job itself ran and failed on the service, and the SDK propagates that as the pipeline's error. Diagnose via the Dataflow console/logs, not the Go stack.","triggerScenarios":"Executing a pipeline whose Dataflow job transitions to JOB_STATE_FAILED — detected during WaitForCompletion polling and returned as the final error of Execute.","commonSituations":"Worker crashes (OOM, quota, image pull failures), user code panics inside DoFns, invalid sources/sinks (bad GCS paths, missing BigQuery datasets), networking/VPC issues preventing workers from reaching services, or insufficient service account permissions.","solutions":["Open the job in the Dataflow console (or `gcloud dataflow jobs describe <jobID>`) and read the failure logs / stack traces","Fix the root cause in the pipeline: worker errors, invalid input/output paths, or quota limits","Check worker logs in Cloud Logging for the failing step and exception","Verify quotas (CPU, in-use IPs, disk) and the worker service account's permissions"],"exampleFix":null,"handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"if err := run(); err != nil && strings.Contains(err.Error(), \"failed\") {\n    var jobID string\n    if n, _ := fmt.Sscanf(err.Error(), \"Job %s failed\", &jobID); n == 1 {\n        log.Printf(\"inspect job at console: %s\", jobID)\n    }\n}","preventionTips":["Set Cloud Logging alerts on job state JOB_STATE_FAILED","Test DoFn logic locally with the direct runner before submitting","Validate input/output paths (GCS, BigQuery) and quotas before launch","Give the worker service account the roles it needs and review stack traces in worker logs"],"tags":["go","dataflow","job-failure","remote-job"],"backgroundTag":"job-state-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}