hashicorp/nomad · error

ack.Error

Error message

ack.Error

What it means

streamingRpcConn converts the ack.Error string returned in the streaming RPC handshake acknowledgment into a Go error and closes the connection. It means the remote side (client or server) rejected the streaming RPC setup and returned an error message in the ack frame.

Solutions

1. Check the ack.Error text (it is the remote's message) and fix the underlying cause
2. Verify both nodes run compatible Nomad versions for the streaming endpoint
3. Ensure the ACL token used has the required capabilities
4. Confirm the remote node/agent is healthy and reachable

conn, err := streamingRpcConn(client, method)
if err != nil {
    // err.Error() carries the remote ack.Error text
    log.Warnf("streaming RPC rejected: %v", err)
    return fallbackToNonStreaming()
}

When it happens

Trigger: Establishing a streaming RPC via RemoteStreamingRpcHandler where the remote peer's handshake ack carries a non-empty Error field — e.g. unknown RPC method, auth/ACL rejection, or the remote node refusing the connection setup.

Common situations: Version skew so the requested streaming endpoint doesn't exist remotely, ACL tokens lacking permission for the streaming RPC, or the remote node being down/draining.

Related errors

• rpc error
• error renewing the lease
• failed to fetch key from any peer
• failed to get conn
• failed to read snapshot metadata

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/0a2f12cb57714676. Report an issue: GitHub.

Appendix: source

Thrown at client/rpc.go:280

	decoder := codec.NewDecoder(conn, structs.MsgpackHandle)
	header := structs.StreamingRpcHeader{
		Method: method,
	}
	if err := encoder.Encode(header); err != nil {
		conn.Close()
		return nil, err
	}

	// Wait for the acknowledgement
	var ack structs.StreamingRpcAck
	if err := decoder.Decode(&ack); err != nil {
		conn.Close()
		return nil, err
	}

	if ack.Error != "" {
		conn.Close()
		return nil, errors.New(ack.Error)
	}

	return conn, nil
}

// setupClientRpc is used to setup the Client's RPC endpoints
func (c *Client) setupClientRpc(rpcs map[string]any) {
	// Create the RPC Server
	c.rpcServer = rpc.NewServer()

	// Initialize the RPC handlers
	if rpcs != nil {
		// override RPCs
		for name, rpc := range rpcs {
			c.rpcServer.RegisterName(name, rpc)
		}
	} else {
		c.endpoints.ClientStats = &ClientStats{c}

View on GitHub (pinned to 482b49bf1a)