{"record":{"id":"ed6d0e8c6ebd2760","repo":"hashicorp/nomad","slug":"csi-controllerexpandvolume-plugin-did-not-return","errorCode":null,"errorMessage":"CSI.ControllerExpandVolume: plugin did not return error or response","messagePattern":"CSI\\.ControllerExpandVolume: plugin did not return error or response","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/csi_endpoint.go","lineNumber":270,"sourceCode":"\n\t// CSI ControllerExpandVolume errors for timeout, codes.Unavailable and\n\t// codes.ResourceExhausted are retried; all other errors are fatal.\n\tcresp, err := plugin.ControllerExpandVolume(ctx, csiReq,\n\t\tgrpc_retry.WithPerRetryTimeout(CSIPluginRequestTimeout),\n\t\tgrpc_retry.WithMax(3),\n\t\tgrpc_retry.WithBackoff(grpc_retry.BackoffExponential(100*time.Millisecond)))\n\tif errors.Is(err, nstructs.ErrCSIClientRPCIgnorable) {\n\t\t// if the volume was deleted out-of-band, we'll get an error from\n\t\t// the plugin but can safely ignore it\n\t\tc.c.logger.Debug(\"could not expand volume\", \"error\", err)\n\t\treturn nil\n\t}\n\tif err != nil {\n\t\treturn fmt.Errorf(\"CSI.ControllerExpandVolume: %v\", err)\n\t}\n\tif cresp == nil {\n\t\tc.c.logger.Warn(\"plugin did not return error or response; this is a bug in the plugin and should be reported to the plugin author\")\n\t\treturn fmt.Errorf(\"CSI.ControllerExpandVolume: plugin did not return error or response\")\n\t}\n\tresp.CapacityBytes = cresp.CapacityBytes\n\tresp.NodeExpansionRequired = cresp.NodeExpansionRequired\n\treturn nil\n}\n\nfunc (c *CSI) ControllerDeleteVolume(req *structs.ClientCSIControllerDeleteVolumeRequest, resp *structs.ClientCSIControllerDeleteVolumeResponse) error {\n\tdefer metrics.MeasureSince([]string{\"client\", \"csi_controller\", \"delete_volume\"}, time.Now())\n\n\tplugin, err := c.findControllerPlugin(req.PluginID)\n\tif err != nil {\n\t\t// the server's view of the plugin health is stale, so let it know it\n\t\t// should retry with another controller instance\n\t\treturn fmt.Errorf(\"CSI.ControllerDeleteVolume: %w: %v\",\n\t\t\tnstructs.ErrCSIClientRPCRetryable, err)\n\t}\n\tdefer plugin.Close()\n","sourceCodeStart":252,"sourceCodeEnd":288,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/csi_endpoint.go#L252-L288","documentation":"In Nomad's CSI client endpoint, after calling the CSI plugin's ControllerExpandVolume RPC, the plugin returned neither an error nor a response object (cresp == nil). The gRPC/Go CSI plugin contract requires one of the two, so this indicates a malformed plugin implementation. Nomad explicitly logs it as a bug in the plugin that should be reported to the plugin author.","triggerScenarios":"A CSI controller plugin (or its client wrapper) returns (nil, nil) from ControllerExpandVolume — e.g. a buggy custom/plugin-in-development implementation, a plugin that silently swallows errors and skips building a response, or a hand-rolled CSI mock plugin.","commonSituations":"Developers testing a custom CSI plugin against Nomad, running an outdated/buggy plugin version that mishandles the expand RPC, or using a stub plugin that returns empty results on ControllerExpandVolume.","solutions":["Report the bug to the CSI plugin vendor/author; the plugin must return either a non-nil ControllerExpandVolumeResponse or a non-nil error","Upgrade the CSI plugin to the latest version where the nil-response bug may be fixed","Check Nomad and plugin version compatibility and restart the plugin task; re-run the volume expansion"],"exampleFix":"// before (buggy plugin code)\nfunc (p *Plugin) ControllerExpandVolume(ctx context.Context, req *csi.ControllerExpandVolumeRequest) (*csi.ControllerExpandVolumeResponse, error) {\n    return nil, nil\n}\n// after\nfunc (p *Plugin) ControllerExpandVolume(ctx context.Context, req *csi.ControllerExpandVolumeRequest) (*csi.ControllerExpandVolumeResponse, error) {\n    resp, err := p.backend.Expand(req.VolumeId, req.CapacityRange.RequiredBytes)\n    if err != nil {\n        return nil, status.Errorf(codes.Internal, \"expand failed: %v\", err)\n    }\n    return &csi.ControllerExpandVolumeResponse{CapacityBytes: resp.Capacity, NodeExpansionRequired: true}, nil\n}","handlingStrategy":"try-catch","validationCode":"// before relying on expand, verify plugin health and implementation\nplug, err := apiClient.Plugins().Get(\"aws-ebs-controller\")\nif err != nil || plug.ControllersHealthy < 1 {\n    log.Fatal(\"controller plugin not healthy; expand would fail\")\n}\nif plug.ControllerInfo.SupportsCondition(structs.CSIControllerSupportsExpandVolume) == false {\n    log.Fatal(\"plugin does not advertise expand support\")\n}","typeGuard":"func isNilPluginResponseError(err error) bool {\n    return err != nil && strings.Contains(err.Error(),\n        \"plugin did not return error or response\")\n}","tryCatchPattern":"err := client.CSI().ControllerExpandVolume(req)\nif err != nil {\n    if isNilPluginResponseError(err) {\n        // not a transient storage failure: report the plugin bug, don't retry\n        reportPluginBug(req.PluginID, err)\n        return err\n    }\n    return err\n}","preventionTips":["Pin to a stable, tested CSI plugin release rather than a dev build","Smoke-test ControllerExpandVolume against the plugin before production use","Report nil-response bugs upstream to the plugin author; keep plugin and Nomad versions compatible"],"tags":["csi","nomad","grpc","plugin-bug"],"backgroundTag":"csi-plugin-nil-response","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}