{"record":{"id":"a7572b22d74591f4","repo":"argoproj/argo-workflows","slug":"plugin-s-save-failed-w","errorCode":null,"errorMessage":"plugin %s save failed: %w","messagePattern":"plugin (.+?) save failed: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"workflow/artifacts/plugin/plugin.go","lineNumber":212,"sourceCode":"\t\t\t\t\twriter.CloseWithError(fmt.Errorf(\"plugin %s stream write failed: %w\", d.pluginName, writeErr))\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}()\n\n\treturn reader, nil\n}\n\n// Save implements ArtifactDriver.Save by calling the plugin service\nfunc (d *Driver) Save(ctx context.Context, path string, outputArtifact *wfv1.Artifact) error {\n\tgrpcArtifact := convertToGRPC(outputArtifact)\n\tresp, err := d.client.Save(ctx, &artifact.SaveArtifactRequest{\n\t\tPath:           path,\n\t\tOutputArtifact: grpcArtifact,\n\t})\n\tif err != nil {\n\t\treturn fmt.Errorf(\"plugin %s save failed: %w\", d.pluginName, err)\n\t}\n\tif !resp.Success {\n\t\treturn fmt.Errorf(\"plugin %s save failed: %s\", d.pluginName, resp.Error)\n\t}\n\treturn nil\n}\n\n// saveStreamChunkSize is the size of each chunk sent over the streaming SaveStream RPC.\n// 2MiB stays well under gRPC's default 4MiB max message size while keeping the\n// per-chunk marshal/syscall overhead low for multi-GB artifacts.\nconst saveStreamChunkSize = 2 * 1024 * 1024\n\n// SaveStream implements ArtifactDriver.SaveStream. If the plugin advertises\n// streaming support (per GetCapabilities), the reader is streamed directly with no\n// local buffering. Otherwise it falls back to buffering to a temp file and calling\n// the existing unary Save, so a plugin that predates streaming keeps working.\n//\n// Capability is checked before reader is touched: once GetCapabilities confirms","sourceCodeStart":194,"sourceCodeEnd":230,"githubUrl":"https://github.com/argoproj/argo-workflows/blob/35bff19146f5a6ada77468c431f2624bd577e373/workflow/artifacts/plugin/plugin.go#L194-L230","documentation":"The unary Save RPC to the artifact plugin failed at the transport/gRPC level. The driver calls client.Save with the artifact path and metadata; any error returned by the gRPC call (network failure, unimplemented method, context timeout, plugin crash) is wrapped as 'plugin %s save failed: %w'. The wrapped gRPC status is the real diagnosis.","triggerScenarios":"Calling Driver.Save (workflow/artifacts/plugin/plugin.go:205) when the plugin pod is down/restarting, the unix socket connection broke, the context deadline expired, or the plugin returns a gRPC error status.","commonSituations":"Plugin OOM-killed during a large artifact save, plugin redeployed mid-workflow, artifact save exceeds gRPC 4MiB message limit for huge inline metadata, or node/network issues on the unix socket.","solutions":["Decode the wrapped gRPC status (status.FromError) to see the exact code (Unavailable, DeadlineExceeded, Unimplemented, ResourceExhausted)","Check the plugin container logs and restart state (kubectl get pods, describe)","If Unavailable, verify the plugin is running and its unix socket path matches configuration","For DeadlineExceeded, increase the controller's timeout or check plugin save performance"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Pre-check plugin connectivity before issuing Save:\nconnState := driver.ConnState() // if exposed, or probe socket:\nif _, err := os.Stat(socketPath); err != nil {\n    return fmt.Errorf(\"plugin socket missing: %w\", err)\n}","typeGuard":"func grpcUnavailable(err error) bool {\n    st, ok := status.FromError(errors.Unwrap(err))\n    return ok && st.Code() == codes.Unavailable\n}","tryCatchPattern":"err := driver.Save(ctx, path, artifact)\nif err != nil {\n    if grpcUnavailable(err) {\n        // backoff and retry: plugin may be restarting\n        return retryWithBackoff(func() error { return driver.Save(ctx, path, artifact) })\n    }\n    return err\n}","preventionTips":["Give plugin pods liveness probes and adequate memory to survive large saves","Pin plugin version with the controller to avoid mid-workflow restarts/redeploys","Use the errors.Unwrap chain and status.FromError to classify the gRPC code before reacting","Set realistic context deadlines matching artifact size"],"tags":["grpc","plugin","artifacts","save"],"backgroundTag":"grpc-unavailable","analyzedSha":"35bff19146f5a6ada77468c431f2624bd577e373","analyzedAt":"2026-09-03T19:34:35.908Z","contentChangedAt":"2026-09-03T19:34:35.908Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}