{"record":{"id":"fb7b92aba5957f58","repo":"hashicorp/nomad","slug":"error-creating-volume-q-with-plugin-q-w-s","errorCode":null,"errorMessage":"error creating volume %q with plugin %q: %w: %s","messagePattern":"error creating volume %q with plugin %q: %w: (.+?)","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/hostvolumemanager/host_volume_plugin.go","lineNumber":363,"sourceCode":"\t\tfmt.Sprintf(\"%s=%d\", EnvCapacityMin, req.RequestedCapacityMinBytes),\n\t\tfmt.Sprintf(\"%s=%d\", EnvCapacityMax, req.RequestedCapacityMaxBytes),\n\t\tfmt.Sprintf(\"%s=%s\", EnvNodeID, req.NodeID),\n\t\tfmt.Sprintf(\"%s=%s\", EnvParameters, params),\n\t}\n\n\tvar pluginResp HostVolumePluginCreateResponse\n\tlog := p.log.With(\"volume_name\", req.Name, \"volume_id\", req.ID)\n\tstdout, _, err := p.runPlugin(ctx, log, \"create\", envVars)\n\tif err != nil {\n\t\tjsonErr := json.Unmarshal(stdout, &pluginResp)\n\t\tif jsonErr != nil {\n\t\t\t// if we got an error, we can't actually count on getting JSON, so\n\t\t\t// optimistically look for it and return the original error\n\t\t\t// otherwise\n\t\t\treturn nil, fmt.Errorf(\n\t\t\t\t\"error creating volume %q with plugin %q: %w\", req.ID, p.ID, err)\n\t\t}\n\t\treturn nil, fmt.Errorf(\"error creating volume %q with plugin %q: %w: %s\",\n\t\t\treq.ID, p.ID, err, pluginResp.Error)\n\t}\n\terr = json.Unmarshal(stdout, &pluginResp)\n\tif err != nil {\n\t\t// note: if a plugin does not return valid json, a volume may be\n\t\t// created without any respective state in Nomad, since we return\n\t\t// an error here after the plugin has done who-knows-what.\n\t\treturn nil, err\n\t}\n\treturn &pluginResp, nil\n}\n\n// Delete calls the executable with the following parameters:\n// arguments: $1=delete\n// environment:\n// - DHV_OPERATION=delete\n// - DHV_CREATED_PATH={path that `create` returned}\n// - DHV_VOLUMES_DIR={directory that volumes should be put in}","sourceCodeStart":345,"sourceCodeEnd":381,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/hostvolumemanager/host_volume_plugin.go#L345-L381","documentation":"Returned by HostVolumePluginExternal.Create when the plugin exits non-zero and DID return parseable JSON, allowing Nomad to append the plugin's own error message (pluginResp.Error) after the wrapped execution error. This is the richer variant of the create failure. Note the code warns that a plugin returning invalid JSON may leave a volume created without Nomad state.","triggerScenarios":"External create plugin exits non-zero but emits valid JSON containing an error field, e.g. it rejected the request: invalid volume ID, filesystem error creating the volume directory, permission denied in VolumesDir, or invalid parameters.","commonSituations":"Target volumes directory doesn't exist or lacks write permission; duplicate volume ID; plugin-side validation rejecting parameters; disk full; plugin compiled against a different response schema.","solutions":["Read the trailing '%s' in the message — it contains the plugin's own error and act on it directly","Check permissions/existence of the plugin's VolumesDir on the client node","Verify volume ID uniqueness and parameter validity for this plugin","Run the plugin manually with NOMAD_OPERATION=create to reproduce","Reconcile Nomad state: a volume may have been created on disk without being registered"],"exampleFix":"// before\nvolume \"web\" { plugin = \"mkdir\" parameters {} }  # plugin error: mkdir: permission denied\n// after\n# on the client node\n$ sudo mkdir -p /opt/nomad/volumes && sudo chown nomad:nomad /opt/nomad/volumes","handlingStrategy":"try-catch","validationCode":"if _, err := os.Stat(volumesDir); err != nil || !writable(volumesDir) {\n    return fmt.Errorf(\"volumes dir %s not writable: %w\", volumesDir, err)\n}\nif _, err := os.Stat(filepath.Join(volumesDir, volID)); err == nil {\n    return fmt.Errorf(\"volume %s already exists\", volID)\n}","typeGuard":"func pluginReportedError(msg string) bool {\n    // message ends with ': <plugin error>' when JSON was parseable\n    return strings.HasSuffix(msg, pluginErrMsg)\n}","tryCatchPattern":"resp, err := plugin.Create(ctx, req)\nif err != nil {\n    parts := strings.Split(err.Error(), \": \")\n    pluginMsg := parts[len(parts)-1]\n    log.Error(\"volume create rejected by plugin\", \"plugin_error\", pluginMsg)\n    switch {\n    case strings.Contains(pluginMsg, \"permission denied\"):\n        // fix ownership and retry\n    case strings.Contains(pluginMsg, \"file exists\"):\n        // deregister/deduplicate ID first\n    }\n    return err\n}","preventionTips":["Pre-create and chown the volumes directory for the nomad user","Use unique volume IDs and check for collisions before Create","Read the trailing plugin error text before generic retries","Manually reproduce with the same env vars the plugin receives"],"tags":["host-volumes","external-plugin","process-execution"],"backgroundTag":"plugin-execution-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}