What causes "query messages for topic %s on all masters: %w" in t8y2/dbx?

Inspect the wrapped cause (%w) to identify the actual per-broker failure and fix it first.

query messages for topic %s on all masters: %w

Error message

query messages for topic %s on all masters: %w

What it means

queryMessagesByKey sends a QueryMessage request to every master broker found in the route. If every request fails, successCount stays 0 and the library wraps the last per-broker error with this message. All masters failed to answer the key-based message query.

Source

Thrown at agents/drivers/rocketmq/messages.go:277

			if message.Queue != nil {
				queueID = message.Queue.QueueId
			}
			messageTopic := message.Topic
			if messageTopic == "" {
				messageTopic = topic
			}
			messages = append(messages, &admin.MessageExt{
				Topic: messageTopic, QueueId: queueID, QueueOffset: message.QueueOffset,
				MsgId: message.MsgId, OffsetMsgId: message.OffsetMsgId, Body: message.Body,
				Flag: int(message.Flag), BornTimestamp: message.BornTimestamp,
				StoreTimestamp: message.StoreTimestamp, BornHost: message.BornHost,
				StoreHost: message.StoreHost, SysFlag: int(message.SysFlag),
				BrokerName: target.BrokerName, Properties: message.GetProperties(),
			})
		}
	}
	if successCount == 0 {
		return nil, fmt.Errorf("query messages for topic %s on all masters: %w", topic, lastErr)
	}
	return messages, nil
}

func buildQueryMessageCommand(topic, key string, maxNum int, beginTimestamp, endTimestamp int64) *remoting.RemotingCommand {
	return remoting.NewRequest(remoting.QueryMessage, map[string]string{
		"topic": topic, "key": key, "maxNum": strconv.Itoa(maxNum),
		"beginTimestamp": strconv.FormatInt(beginTimestamp, 10),
		"endTimestamp":   strconv.FormatInt(endTimestamp, 10),
	})
}

func (a *rocketMQAgent) queryMessageByTopic(params map[string]any) (any, error) {
	topic, err := requireString(params, "topic")
	if err != nil {
		return nil, err
	}
	client, config, _ := a.requireClient()

View on GitHub (pinned to c0390bff16)

Solutions

Inspect the wrapped cause (%w) to identify the actual per-broker failure and fix it first.
Test connectivity: nc -vz <brokerIP> 10911 from the client host.
Increase config.ConnectTimeout / RequestTimeout if brokers are slow or remote.
Confirm masters are alive (mqadmin clusterList) and restart any crashed ones.
If requests are rejected, check broker permissions and that the topic perm includes the READ bit (2, typical value 6).

Defensive patterns

Strategy: retry

Validate before calling

// Pre-check that each master address is reachable before issuing the query.
for _, bd := range route.BrokerDatas {
    if addr := bd.BrokerAddrs["0"]; addr != "" {
        conn, err := net.DialTimeout("tcp", hostPort(addr), 2*time.Second)
        if err != nil { log.Printf("master %s unreachable: %v", bd.BrokerName, err) } else { conn.Close() }
    }
}

Try / catch

msgs, err := queryMessageByKey(params)
if err != nil {
    var netErr net.Error
    if errors.As(err, &netErr) && netErr.Timeout() {
        return retryWithBackoff(3, func() error { return queryMessageByKey(params) })
    }
    return fmt.Errorf("key query failed on all masters: %w", err)
}

Prevention

Always inspect the wrapped cause (%w) — it names the actual broker failure.
Set ConnectTimeout/RequestTimeout generously for remote or busy clusters.
Keep firewall rules open for broker listen ports (default 10911).
Alert on broker downtime; query-by-key is unavailable while masters are down.

When it happens

Trigger: Calling viewMessage / queryMessageByKey when every master broker request fails due to network errors, timeouts, or remoting rejections — broker down, connection refused, firewall, or unsupported response code.

Common situations: Firewall blocking the broker listen port (default 10911); all brokers down while NameServer returns cached routes; connectTimeout too small; TLS/plain-text mismatch; broker overloaded and dropping requests.

Related errors

AI-assisted analysis of t8y2/dbx@c0390bff16 (2026-09-05). Data as JSON: /api/errors/6002fd718fbb502c. Report an issue: GitHub.