t8y2/dbx · critical

handshake %s: %w

Error message

handshake %s: %w

What it means

This panic wraps a failure of the JSON-RPC 'handshake' call issued right after a measured agent startup. Like the warm-up variant, it fires when the agent process died or misbehaved between the ready line and the handshake response: stdout closed early ('agent response unavailable'), response id mismatch, JSON decode error, or an agent-returned JSON-RPC error. The harness panics because a failed handshake invalidates that startup sample and usually indicates a systematically broken agent.

Source

Thrown at agents/drivers/rabbitmq/bench/agent_compare.go:179

	readySamples := map[string][]float64{}
	handshakeSamples := map[string][]float64{}
	rssSamples := map[string][]int64{}
	readyDurations := map[string]time.Duration{}
	handshakeDurations := map[string]time.Duration{}
	for iteration := 0; iteration < iterations; iteration++ {
		order := agents
		if iteration%2 == 1 {
			order = []agentSpec{agents[1], agents[0]}
		}
		for _, agent := range order {
			process, readyDuration, err := startAgent(agent.Command)
			if err != nil {
				panic(fmt.Errorf("start %s: %w", agent.Name, err))
			}
			handshakeStart := time.Now()
			if _, err := process.call("handshake", map[string]any{}); err != nil {
				process.kill()
				panic(fmt.Errorf("handshake %s: %w", agent.Name, err))
			}
			handshakeDuration := time.Since(handshakeStart)
			readySamples[agent.Name] = append(readySamples[agent.Name], milliseconds(readyDuration))
			handshakeSamples[agent.Name] = append(
				handshakeSamples[agent.Name],
				milliseconds(readyDuration+handshakeDuration),
			)
			rssSamples[agent.Name] = append(rssSamples[agent.Name], readRSSKB(process.command.Process.Pid))
			readyDurations[agent.Name] += readyDuration
			handshakeDurations[agent.Name] += readyDuration + handshakeDuration
			if err := process.close(); err != nil {
				panic(fmt.Errorf("close %s: %w", agent.Name, err))
			}
		}
	}

	results := make([]benchmarkResult, 0, len(agents)*2)
	for _, agent := range agents {

View on GitHub (pinned to c0390bff16)

Solutions

  1. Inspect agent stderr for the crash/error at handshake time.
  2. Ensure the agent logs only to stderr, never stdout (stdout is the JSON-RPC channel) — stray log lines break JSON decoding.
  3. Rebuild both agents against the same handshake protocol so ids and envelope fields match.
  4. Test the agent standalone: pipe a handshake request to its stdin and confirm a well-formed {"id":1,"result":...} reply.
  5. Check memory limits; JVM agents can be OOM-killed during startup.
  6. Replace panic with an error return and skip/retry that sample instead of aborting.

Example fix

// before
if _, err := process.call("handshake", map[string]any{}); err != nil {
	process.kill()
	panic(fmt.Errorf("handshake %s: %w", agent.Name, err))
}
// after
if _, err := process.call("handshake", map[string]any{}); err != nil {
	process.kill()
	return nil, fmt.Errorf("handshake %s: %w", agent.Name, err)
}
Defensive patterns

Strategy: try-catch

Validate before calling

// After startAgent, confirm the process is still alive before handshaking
if process.command.ProcessState != nil {
	log.Fatal("agent exited before handshake")
}
// And smoke-test handshake offline once:
// echo '{"jsonrpc":"2.0","id":1,"method":"handshake","params":{}}' | <agent>

Type guard

func isProtocolError(resp *agentResponse) bool {
	return resp != nil && resp.Error != nil
}
func idMatches(resp *agentResponse, want int64) bool {
	return resp != nil && resp.ID == want
}

Try / catch

func safeHandshake(p *agentProcess, name string) (err error) {
	defer func() {
		if r := recover(); r != nil {
			p.kill()
			err = fmt.Errorf("handshake %s: %v", name, r)
		}
	}()
	_, err = p.call("handshake", map[string]any{})
	return err
}
// On error: kill the child, capture stderr, and retry once before failing the run.

Prevention

When it happens

Trigger: In benchmarkStartups measured iterations: after startAgent() succeeded, process.call("handshake", map[string]any{}) returns an error — child exited after becoming ready, scanner EOF/reader error on the response, unmarshalling failure, response.ID != nextID, or response.Error != nil.

Common situations: Agent crashes on first real request (dependency missing, DB/broker connection refused inside agent init); protocol version mismatch after rebuilding one agent (id or envelope format changed); JVM OOM during startup under benchmark load; agent writes extra non-JSON log lines to stdout, corrupting the JSON-RPC stream so unmarshal fails.

Understand the failure class

Related errors


AI-assisted analysis of t8y2/dbx@c0390bff16 (2026-09-05). Data as JSON: /api/errors/53c0dc64fd661ca9. Report an issue: GitHub.