{"record":{"id":"a0a20872a4527b48","repo":"hashicorp/nomad","slug":"could-not-validate-task-driver-capabilities-v","errorCode":null,"errorMessage":"could not validate task driver capabilities: %v","messagePattern":"could not validate task driver capabilities: (.+?)","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/allocrunner/csi_hook.go","lineNumber":230,"sourceCode":"}\n\ntype volumePublishResult struct {\n\trequest        *structs.VolumeRequest // the request from the jobspec\n\tvolume         *structs.CSIVolume     // the volume we get back from the server\n\tpublishContext map[string]string      // populated after claim if provided by plugin\n\tstub           *state.CSIVolumeStub   // populated from volume, plugin, or stub\n}\n\n// validateTasksSupportCSI verifies that at least one task in the group uses a\n// task driver that supports CSI. This prevents us from publishing CSI volumes\n// only to find out once we get to the taskrunner/volume_hook that no task can\n// mount them.\nfunc (c *csiHook) validateTasksSupportCSI(tg *structs.TaskGroup) error {\n\n\tfor _, task := range tg.Tasks {\n\t\tcaps, err := c.allocRunnerShim.GetTaskDriverCapabilities(task.Name)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"could not validate task driver capabilities: %v\", err)\n\t\t}\n\n\t\tif caps.MountConfigs == drivers.MountConfigSupportNone {\n\t\t\tcontinue\n\t\t}\n\n\t\treturn nil\n\t}\n\n\treturn fmt.Errorf(\"no task supports CSI\")\n}\n\n// restoreMounts tries to restore the mount info from the local client state and\n// then verifies it with the plugin. If the volume is already mounted, we don't\n// want to re-run the claim and mount workflow again. This lets us tolerate\n// restarting clients even on disconnected nodes.\nfunc (c *csiHook) restoreMounts(results map[string]*volumePublishResult) error {\n\tstubs, err := c.allocRunnerShim.GetCSIVolumes()","sourceCodeStart":212,"sourceCodeEnd":248,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/allocrunner/csi_hook.go#L212-L248","documentation":"Before using CSI volumes, the hook validates that each task's driver advertises CSI capability support via GetTaskDriverCapabilities. This error means the capability query to the task driver itself failed, so Nomad cannot determine whether the driver can mount CSI configs, and prerun aborts.","triggerScenarios":"csiHook.validateTasksSupportCSI (called from Prerun) -> c.allocRunnerShim.GetTaskDriverCapabilities(task.Name) returns an error: the driver plugin is not running, failed to initialize, or the capability RPC failed.","commonSituations":"Docker/exec/podman driver plugin crashed or failed to start on the client; driver missing from the client's driver configuration; Nomad client version newer than driver plugin (missing Capabilities RPC during mixed-version upgrades); plugin handshake failure after node restart.","solutions":["Check the driver is loaded and healthy: nomad node status <node> (driver health/attributes)","Restart the Nomad client agent so the driver plugin re-handshakes","Check client logs for driver plugin startup errors and fix the driver config/plugin install","Upgrade the external driver plugin (e.g. podman) to a version that implements driver capabilities, and keep Nomad client/plugin versions compatible"],"exampleFix":"// before\n# nomad node status: docker driver = unhealthy\n// after\n$ sudo systemctl restart nomad\n$ nomad node status <node>  # docker: healthy, capabilities detected","handlingStrategy":"validation","validationCode":"const node = await nomad.get('node/self');\nfor (const driver of ['docker', 'exec']) {\n  const d = node.Drivers?.[driver];\n  if (!d?.Detected || !d?.Healthy) throw new Error(`driver ${driver} not healthy`);\n}","typeGuard":"function driverIsHealthy(d) {\n  return !!d && typeof d === 'object' && d.Detected === true && d.Healthy === true;\n}","tryCatchPattern":"try {\n  await runJob(job);\n} catch (err) {\n  if (String(err).includes('task driver capabilities')) {\n    await restartNomadAgent(nodeId); // re-handshake driver plugins\n    await waitForDriverHealthy(nodeId, 'docker');\n    await runJob(job);\n  } else throw err;\n}","preventionTips":["Monitor nomad node status driver Detected/Healthy fields","Keep external driver plugins updated to versions implementing the Capabilities RPC","Restart clients after upgrades so plugins re-handshake before scheduling CSI jobs","Alert on driver plugin crashes in Nomad client logs"],"tags":["csi","drivers","nomad","plugin"],"backgroundTag":"task-driver-capabilities-unavailable","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}