{"record":{"id":"2d4c92899bb46d9f","repo":"hashicorp/nomad","slug":"missing-region-for-target-rpc","errorCode":null,"errorMessage":"missing region for target RPC","messagePattern":"missing region for target RPC","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/rpc.go","lineNumber":588,"sourceCode":"\t\tcase pool.RpcNomad:\n\t\t\tgo r.handleNomadConn(ctx, sub, rpcServer)\n\t\tcase pool.RpcStreaming:\n\t\t\tgo r.handleStreamingConn(sub)\n\n\t\tdefault:\n\t\t\tr.logger.Error(\"multiplex_v2 unrecognized first RPC byte\", \"byte\", buf[0])\n\t\t\treturn\n\t\t}\n\t}\n\n}\n\n// forward is used to forward to a remote region or to forward to the local leader\n// Returns a bool of if forwarding was performed, as well as any error\nfunc (r *rpcHandler) forward(method string, info structs.RPCInfo, args any, reply any) (bool, error) {\n\tregion := info.RequestRegion()\n\tif region == \"\" {\n\t\treturn true, fmt.Errorf(\"missing region for target RPC\")\n\t}\n\n\t// Handle region forwarding\n\tif region != r.srv.config.Region {\n\t\t// Mark that we are forwarding the RPC\n\t\tinfo.SetForwarded()\n\t\terr := r.forwardRegion(region, method, args, reply)\n\t\treturn true, err\n\t}\n\n\t// Check if we can allow a stale read\n\tif info.IsRead() && info.AllowStaleRead() {\n\t\treturn false, nil\n\t}\n\n\tremoteServer, err := r.getLeaderForRPC()\n\tif err != nil {\n\t\treturn true, err","sourceCodeStart":570,"sourceCodeEnd":606,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/rpc.go#L570-L606","documentation":"forward() requires every RPC to carry a target region via RPCInfo.RequestRegion(). An empty region means the request struct did not have its Region field set (or the RPCInfo implementation failed to provide it), so the handler cannot decide whether to serve locally or forward. This is an internal invariant violation rather than a user-facing condition.","triggerScenarios":"An RPC request struct passed to forward() with an empty Region field — custom/forked RPC handlers, test code constructing bare request structs, or an RPC type missing SetRegion/RequestRegion wiring.","commonSituations":"Hand-built RPC calls in tests or tools against the Nomad internal API; new RPC endpoints added in forks that forget to set Region; version drift where a new endpoint is called against an older server lacking region handling.","solutions":["Set the Region field on the request struct before dispatching the RPC","Ensure the RPC type implements structs.RPCInfo (RequestRegion/SetRegion)","Check both server and client are on compatible Nomad versions","Fix custom/test code that builds requests without a region"],"exampleFix":"// before\nreq := &structs.JobListRequest{}\nsrv.RPC(\"Job.List\", req, &reply)\n// after\nreq := &structs.QueryOptions{Region: \"us-east-1\"}\nsrv.RPC(\"Job.List\", req, &reply)","handlingStrategy":"validation","validationCode":"if req.RequestRegion() == \"\" {\n    return fmt.Errorf(\"request Region must be set before RPC dispatch\")\n}","typeGuard":null,"tryCatchPattern":"if _, err := srv.RPC(method, req, reply); err != nil && strings.Contains(err.Error(), \"missing region\") {\n    return fmt.Errorf(\"caller bug: %s request had no Region set: %w\", method, err)\n}","preventionTips":["Always set Region (via QueryOptions/WriteRequest) on internal RPCs","Implement structs.RPCInfo on any custom request types","Test custom RPC handlers with a populated region field"],"tags":["nomad","rpc","forwarding","internal-api"],"backgroundTag":"missing-rpc-region","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}