{"record":{"id":"91ef07c78a9ebb6f","repo":"hashicorp/nomad","slug":"node-plugin-returned-an-error-v","errorCode":null,"errorMessage":"node plugin returned an error: %v","messagePattern":"node plugin returned an error: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"plugins/csi/client.go","lineNumber":953,"sourceCode":"\t\tcode := status.Code(err)\n\t\tswitch code {\n\t\tcase codes.InvalidArgument:\n\t\t\treturn nil, fmt.Errorf(\n\t\t\t\t\"requested capabilities not compatible with volume %q: %v\",\n\t\t\t\treq.ExternalVolumeID, err)\n\t\tcase codes.NotFound:\n\t\t\treturn nil, fmt.Errorf(\"%w: volume %q could not be found: %v\",\n\t\t\t\tstructs.ErrCSIClientRPCIgnorable, req.ExternalVolumeID, err)\n\t\tcase codes.FailedPrecondition:\n\t\t\treturn nil, fmt.Errorf(\"volume %q cannot be expanded while in use: %v\", req.ExternalVolumeID, err)\n\t\tcase codes.OutOfRange:\n\t\t\treturn nil, fmt.Errorf(\n\t\t\t\t\"unsupported capacity_range for volume %q: %v\", req.ExternalVolumeID, err)\n\t\tcase codes.Internal:\n\t\t\treturn nil, fmt.Errorf(\n\t\t\t\t\"node plugin returned an internal error, check the plugin allocation logs for more information: %v\", err)\n\t\tdefault:\n\t\t\treturn nil, fmt.Errorf(\"node plugin returned an error: %v\", err)\n\t\t}\n\t}\n\n\treturn &NodeExpandVolumeResponse{resp.GetCapacityBytes()}, nil\n}\n","sourceCodeStart":935,"sourceCodeEnd":959,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/plugins/csi/client.go#L935-L959","documentation":"The catch-all branch of NodeExpandVolume error handling in Nomad's CSI client. Any gRPC status code not explicitly mapped (e.g. Unavailable, DeadlineExceeded, Unknown) is surfaced as 'node plugin returned an error'. It is a generic pass-through of the plugin error from the volume expansion RPC.","triggerScenarios":"NodeExpandVolume invoked and the plugin returns any non-special-cased gRPC status: plugin temporarily unavailable, context deadline exceeded, unknown plugin error.","commonSituations":"CSI plugin task restarting or temporarily down on the node; network partition between Nomad client and plugin socket; plugin reporting vendor-specific errors with uncommon codes.","solutions":["Check plugin health on the node (`nomad plugin status`, plugin alloc state) and retry once it is running — transient codes like Unavailable often resolve","Inspect the plugin allocation logs for the underlying error detail (%v contains the original gRPC error)","If persistent, redeploy/upgrade the CSI plugin and confirm the volume's node is correctly registered"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// shell: ensure the node plugin task is running before expanding\nnomad node status <node-id> -verbose | grep -i csi\nnomad plugin status","typeGuard":null,"tryCatchPattern":"// Go: retry transient plugin failures with backoff\nerr := retry.Do(func() error {\n    _, err := nodeClient.NodeExpandVolume(ctx, req)\n    return err\n}, retry.Attempts(3), retry.Delay(time.Second))","preventionTips":["Retry expansion on transient plugin unavailability with capped backoff","Alert on CSI plugin restarts/crash loops","Capture the wrapped %v error and correlate it with plugin alloc logs for diagnosis"],"tags":["csi","storage","grpc","plugin"],"backgroundTag":"csi-plugin-rpc-error","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}