{"record":{"id":"fa3bba8ced2e3053","repo":"hashicorp/nomad","slug":"controller-create-snapshot-v","errorCode":null,"errorMessage":"controller create snapshot: %v","messagePattern":"controller create snapshot: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/client_csi_endpoint.go","lineNumber":140,"sourceCode":"\t\t\"ClientCSI.ControllerListVolumes\",\n\t\tstructs.RateMetricList,\n\t\targs, reply)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"controller list volumes: %v\", err)\n\t}\n\treturn nil\n}\n\nfunc (a *ClientCSI) ControllerCreateSnapshot(args *cstructs.ClientCSIControllerCreateSnapshotRequest, reply *cstructs.ClientCSIControllerCreateSnapshotResponse) error {\n\tdefer metrics.MeasureSince([]string{\"nomad\", \"client_csi_controller\", \"create_snapshot\"}, time.Now())\n\n\terr := a.sendCSIControllerRPC(args.PluginID,\n\t\t\"CSI.ControllerCreateSnapshot\",\n\t\t\"ClientCSI.ControllerCreateSnapshot\",\n\t\tstructs.RateMetricWrite,\n\t\targs, reply)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"controller create snapshot: %v\", err)\n\t}\n\treturn nil\n}\n\nfunc (a *ClientCSI) ControllerDeleteSnapshot(args *cstructs.ClientCSIControllerDeleteSnapshotRequest, reply *cstructs.ClientCSIControllerDeleteSnapshotResponse) error {\n\tdefer metrics.MeasureSince([]string{\"nomad\", \"client_csi_controller\", \"delete_snapshot\"}, time.Now())\n\n\terr := a.sendCSIControllerRPC(args.PluginID,\n\t\t\"CSI.ControllerDeleteSnapshot\",\n\t\t\"ClientCSI.ControllerDeleteSnapshot\",\n\t\tstructs.RateMetricWrite,\n\t\targs, reply)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"controller delete snapshot: %v\", err)\n\t}\n\treturn nil\n}\n","sourceCodeStart":122,"sourceCodeEnd":158,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/client_csi_endpoint.go#L122-L158","documentation":"Nomad's ClientCSI.ControllerCreateSnapshot forwards a CSI.ControllerCreateSnapshot RPC to a healthy controller plugin instance via sendCSIControllerRPC. On any forwarding or plugin RPC failure the error is wrapped as 'controller create snapshot: %v'. The snapshot was not created.","triggerScenarios":"Calling ControllerCreateSnapshot (e.g. `nomad volume snapshot create`) when the controller plugin is missing, unhealthy, unreachable, or the CSI plugin rejects the snapshot request (unsupported volume, driver error).","commonSituations":"Controller plugin job not running; volume's pluginID mismatched; storage backend rejecting snapshot (volume in use, quota exceeded); server-to-client RPC timeout.","solutions":["Run `nomad plugin status <pluginID>` and confirm a healthy controller instance exists.","Ensure the controller plugin job is running and connected (`nomad node status` shows the client).","Check the CSI driver logs for backend-side snapshot failures (permissions, quotas, driver limits).","Retry the snapshot command once connectivity/health is restored."],"exampleFix":"# before: controller job stopped\nnomad job stop aws-ebs-csi-controller\n# after: redeploy and verify health before snapshotting\nnomad job run aws-ebs-csi-controller.nomad.hcl\nnomad plugin status aws-ebs-controller\nnomad volume snapshot create ebs-vol my-snapshot","handlingStrategy":"retry","validationCode":"// verify plugin health and volume existence first\np, _, _ := client.Plugins().Info(ctx, pluginID, nil)\nv, _, _ := client.Volumes().Info(ctx, volID, nil)\nif p.ControllersHealthy == 0 || v == nil {\n    return fmt.Errorf(\"cannot snapshot: plugin or volume not ready\")\n}","typeGuard":"func canSnapshot(p *api.CSIPlugin, v *api.CSIVolume) bool {\n    return p != nil && p.ControllersHealthy > 0 && v != nil\n}","tryCatchPattern":"err := csi.ControllerCreateSnapshot(args, reply)\nif err != nil {\n    if strings.Contains(err.Error(), \"controller create snapshot:\") {\n        // check plugin logs, retry after health restored\n    }\n    return err\n}","preventionTips":["Verify controller health before snapshot workflows.","Check backend quotas/permissions before creating snapshots.","Retry transient failures with backoff.","Monitor plugin job stability."],"tags":["csi","nomad","snapshot","rpc-forwarding"],"backgroundTag":"csi-controller-unavailable","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}