{"record":{"id":"c68066e089a74579","repo":"hashicorp/nomad","slug":"could-not-query-host-volume-w","errorCode":null,"errorMessage":"could not query host volume: %w","messagePattern":"could not query host volume: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/host_volume_endpoint.go","lineNumber":696,"sourceCode":"\tif !allowVolume(aclObj, args.RequestNamespace()) {\n\t\treturn structs.ErrPermissionDenied\n\t}\n\n\tif args.VolumeID == \"\" {\n\t\treturn fmt.Errorf(\"missing volume ID to delete\")\n\t}\n\n\tsnap, err := v.srv.State().Snapshot()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tns := args.RequestNamespace()\n\tid := args.VolumeID\n\n\tvol, err := snap.HostVolumeByID(nil, ns, id, true)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"could not query host volume: %w\", err)\n\t}\n\tif vol == nil {\n\t\treturn fmt.Errorf(\"no such volume: %s\", id)\n\t}\n\tif len(vol.Allocations) > 0 {\n\t\tallocIDs := helper.ConvertSlice(vol.Allocations,\n\t\t\tfunc(a *structs.AllocListStub) string { return a.ID })\n\t\treturn fmt.Errorf(\"volume %s in use by allocations: %v\", id, allocIDs)\n\t}\n\n\t// serialize client RPC and raft write per volume ID\n\tindex, err := v.serializeCall(vol.ID, \"delete\", func() (uint64, error) {\n\t\tif err := v.deleteVolume(vol); err != nil {\n\t\t\tif structs.IsErrUnknownNode(err) || structs.IsErrNoNodeConn(err) {\n\t\t\t\tif !args.Force {\n\t\t\t\t\treturn 0, fmt.Errorf(\n\t\t\t\t\t\t\"volume cannot be removed from unknown node without force=true\")\n\t\t\t\t}","sourceCodeStart":678,"sourceCodeEnd":714,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/host_volume_endpoint.go#L678-L714","documentation":"Delete looks up the volume via snap.HostVolumeByID; if the state store query itself returns an error (rather than a nil volume), the server wraps it with 'could not query host volume'. This indicates an internal state-store failure, not a missing volume — the distinct 'no such volume' error handles the nil case.","triggerScenarios":"A memdb/state-store error while reading HostVolumeByID during Delete — corrupted state, snapshot failure, or low-level store errors; not caused by a nonexistent volume (that yields 'no such volume' instead).","commonSituations":"State store instability after crashes; disk I/O issues on the server; rare internal errors during leader transitions.","solutions":["Retry the delete — transient state errors often resolve on the next attempt or after leadership stabilizes.","Check server logs around the error for the underlying state-store cause.","If persistent, inspect server health/disk and consider restarting the unhealthy server or restoring from a known-good state."],"exampleFix":"// client-side resilience\nerr := client.HostVolumes().Delete(ns, id, nil)\nif err != nil && strings.Contains(err.Error(), \"could not query host volume\") {\n    time.Sleep(2 * time.Second)\n    err = client.HostVolumes().Delete(ns, id, nil) // retry transient state error\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"err := client.HostVolumes().Delete(ns, id, nil)\nif err != nil && strings.Contains(err.Error(), \"could not query host volume\") {\n    // transient state-store error; bounded retry\n    for i := 0; i < 3; i++ {\n        time.Sleep(time.Duration(1<<i) * time.Second)\n        if err = client.HostVolumes().Delete(ns, id, nil); err == nil || !strings.Contains(err.Error(), \"could not query host volume\") { break }\n    }\n}","preventionTips":["Monitor server health and state store stability.","Avoid volume deletions immediately after leader elections.","Alert on wrapped state-store errors in server logs."],"tags":["nomad","host-volumes","state-store","internal-error"],"backgroundTag":"state-store-query-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}