shadow1ng/fscan · error

kafka response too large: %d

Error message

kafka response too large: %d

What it means

kafkaRecv rejects a declared response length exceeding maxKafkaResponseSize to prevent unbounded memory allocation from hostile or desynced responses. msg := make([]byte, msgLen) would otherwise allocate attacker-controlled memory. This is a defensive DoS guard on the framing parser.

Source

Thrown at plugins/services/kafka.go:194

	copy(buf[14:], clientID)
	copy(buf[14+len(clientID):], body)

	_, err := conn.Write(buf)
	return err
}

func kafkaRecv(conn io.Reader) ([]byte, error) {
	// 读取 4 字节长度
	lenBuf := make([]byte, 4)
	if _, err := io.ReadFull(conn, lenBuf); err != nil {
		return nil, err
	}
	msgLen := int(binary.BigEndian.Uint32(lenBuf))
	if msgLen < 4 {
		return nil, fmt.Errorf("invalid kafka response length: %d", msgLen)
	}
	if msgLen > maxKafkaResponseSize {
		return nil, fmt.Errorf("kafka response too large: %d", msgLen)
	}
	// 读取消息体
	msg := make([]byte, msgLen)
	if _, err := io.ReadFull(conn, msg); err != nil {
		return nil, err
	}
	// 跳过 correlation_id (4B)，返回 body
	return msg[4:], nil
}

func kafkaString(s string) []byte {
	b := []byte(s)
	buf := make([]byte, 2+len(b))
	binary.BigEndian.PutUint16(buf, uint16(len(b)))
	copy(buf[2:], b)
	return buf
}

View on GitHub (pinned to 95cc12e753)

Solutions

1. If legitimate large Metadata responses hit the limit, raise maxKafkaResponseSize appropriately
2. Reconnect on this error — the stream is almost certainly desynced and unrecoverable
3. Verify framing alignment after every read; never continue reading after a size rejection
4. Use io.LimitReader-style allocation caps even if the limit is raised

Example fix

// before
if msgLen > maxKafkaResponseSize {
    return nil, fmt.Errorf("kafka response too large: %d", msgLen)
}
// after
if msgLen > maxKafkaResponseSize {
    return nil, fmt.Errorf("kafka response too large: %d (limit %d); stream likely desynced, reconnect required", msgLen, maxKafkaResponseSize)
}

// Cap accepted frame size at the same constant used by the parser
const expectedMax = maxKafkaResponseSize
// ensure your broker's Metadata responses (topic count) stay well under expectedMax

func isPlausibleKafkaLen(lenBuf []byte) bool {
    l := binary.BigEndian.Uint32(lenBuf)
    return l >= 4 && l <= maxKafkaResponseSize
}

resp, err := kafkaRecv(conn)
if err != nil {
    if strings.Contains(err.Error(), "kafka response too large") {
        log.Println("oversized kafka frame; stream desynced or hostile peer — reconnect")
        return
    }
    return err
}

When it happens

Trigger: kafkaRecv (called by doKafkaAuth, identifyService, tests): the 4-byte length prefix decodes to a value greater than maxKafkaResponseSize — desynced stream reading a payload as a length, a malicious peer, or a genuinely oversized response the limit doesn't accommodate.

Common situations: Stream misalignment after an earlier short read causing payload bytes to be interpreted as a length prefix; scanning a honeypot/service that sends huge random frames; maxKafkaResponseSize set too low for brokers returning large Metadata responses (many topics).

Understand the failure class

Background: payload too large / request exceeds maximum size: why libraries cap bytes and how to fix oversize payloads — this error's family across 50 libraries.

Related errors

• invalid kafka response length: %d
• %s [socks5_unsupported_version]
• socks5_invalid_request
• SASL handshake error: %d
• invalid SCRAM server-first payload

AI-assisted analysis of shadow1ng/fscan@95cc12e753 (2026-09-06). Data as JSON: /api/errors/93a5114307d03fb1. Report an issue: GitHub.