{"record":{"id":"dd442c0351fd8d0c","repo":"t8y2/dbx","slug":"zookeeper-response-frame-is-d-bytes-maximum-is-dd442c","errorCode":null,"errorMessage":"ZooKeeper response frame is %d bytes, maximum is %d","messagePattern":"ZooKeeper response frame is (.+?) bytes, maximum is (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"agents/drivers/hive-go/zookeeper_protocol.go","lineNumber":363,"sourceCode":"\theader := make([]byte, 4)\n\tbinary.BigEndian.PutUint32(header, uint32(len(payload)))\n\tif err := writeAll(client.connection, header); err != nil {\n\t\treturn err\n\t}\n\treturn writeAll(client.connection, payload)\n}\n\nfunc (client *protocolZooKeeperClient) readFrame() ([]byte, error) {\n\tif err := client.connection.SetReadDeadline(time.Now().Add(client.timeout)); err != nil {\n\t\treturn nil, err\n\t}\n\theader := make([]byte, 4)\n\tif _, err := io.ReadFull(client.connection, header); err != nil {\n\t\treturn nil, err\n\t}\n\tlength := int(binary.BigEndian.Uint32(header))\n\tif length < 0 || length > zooKeeperMaxFrameSize {\n\t\treturn nil, fmt.Errorf(\"ZooKeeper response frame is %d bytes, maximum is %d\", length, zooKeeperMaxFrameSize)\n\t}\n\tpayload := make([]byte, length)\n\tif _, err := io.ReadFull(client.connection, payload); err != nil {\n\t\treturn nil, err\n\t}\n\treturn payload, nil\n}\n\nfunc writeAll(writer io.Writer, payload []byte) error {\n\tfor len(payload) > 0 {\n\t\twritten, err := writer.Write(payload)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif written <= 0 {\n\t\t\treturn io.ErrShortWrite\n\t\t}\n\t\tpayload = payload[written:]","sourceCodeStart":345,"sourceCodeEnd":381,"githubUrl":"https://github.com/t8y2/dbx/blob/c0390bff16418b651f4728520d99adf8ce48829a/agents/drivers/hive-go/zookeeper_protocol.go#L345-L381","documentation":"readFrame reads a 4-byte big-endian length prefix from the ZooKeeper connection, then reads exactly that many payload bytes. If the declared frame length is negative or exceeds zooKeeperMaxFrameSize, the client refuses to allocate that buffer and returns this error instead. This protects the client from corrupted responses and from malicious/broken servers that would otherwise trigger huge or impossible allocations.","triggerScenarios":"Any call through request (i.e. any ZooKeeper operation via newProtocolZooKeeperClient) when the server (or an intermediary proxy) sends a frame whose 4-byte length header decodes to a value above zooKeeperMaxFrameSize or negative; also hit by readZooKeeperTestFrame when a test frame header is malformed.","commonSituations":"Responses larger than the library's cap (e.g. a huge getChildren or getData reply), connecting to a non-ZooKeeper service on the configured port so random bytes are interpreted as a length header, a proxy/load balancer truncating or mangling TCP streams, or protocol desync after a partial read on a reused connection.","solutions":["Verify the connection actually points at a ZooKeeper server (correct host/port, no generic TCP proxy echoing garbage).","Reduce the response size: narrow the query (smaller result sets, use Children instead of large getData payloads) so the reply fits under the frame cap.","Check the network path (TLS termination, HAProxy) for stream corruption or truncation; enable TLS on both ends if required.","If legitimately huge responses are expected, raise zooKeeperMaxFrameSize and redeploy, accepting the larger memory footprint.","Re-establish the connection after the error; protocol state may be desynced and further reads will fail."],"exampleFix":"// before\nclient, _ := hive.Dial(\"zk://10.0.0.1:9999\") // wrong port: not ZooKeeper\nchildren, err := client.Children(\"/a/very/large/znode\")\n// after\nclient, _ := hive.Dial(\"zk://10.0.0.1:2181\") // correct ZooKeeper port\nchildren, err := client.Children(\"/a/large/znode\") // or fetch in smaller chunks","handlingStrategy":"validation","validationCode":"// before dialing, sanity-check the endpoint is a ZooKeeper server\nconn, err := net.DialTimeout(\"tcp\", \"10.0.0.1:2181\", 5*time.Second)\nif err != nil { log.Fatal(err) }\nconn.SetDeadline(time.Now().Add(3 * time.Second))\nconn.Write([]byte(\"ruok\"))\nbuf := make([]byte, 8)\nn, _ := conn.Read(buf)\nif string(buf[:n]) != \"imok\" { log.Fatal(\"endpoint is not ZooKeeper\") }\nconn.Close()","typeGuard":"func isFrameTooLarge(err error) bool {\n\treturn err != nil && strings.Contains(err.Error(), \"maximum is\")\n}","tryCatchPattern":"children, err := client.Children(path)\nif err != nil {\n\tif isFrameTooLarge(err) {\n\t\t// reconnect (stream is desynced) and fall back to a smaller query\n\t\tclient.Close()\n\t\tclient, err = hive.Dial(connStr)\n\t\tif err == nil {\n\t\t\tchildren, err = client.Children(shardPath(path))\n\t\t}\n\t}\n\treturn fmt.Errorf(\"zk children: %w\", err)\n}","preventionTips":["Always point the client at a real ZooKeeper port (2181) and verify with the 'ruok' health check before use.","Keep znode payloads and directory listings small so replies stay far below the frame cap.","Avoid generic TCP proxies on the ZooKeeper path; use ones that preserve the byte stream if unavoidable.","Recreate the client connection after any decode error instead of retrying on the same socket.","Alert on this error in production — it usually means stream corruption, not a transient failure."],"tags":["zookeeper","network","frame-size-limit","protocol-decode"],"backgroundTag":"response-frame-size-exceeded","analyzedSha":"c0390bff16418b651f4728520d99adf8ce48829a","analyzedAt":"2026-09-05T23:05:10.900Z","contentChangedAt":"2026-09-05T23:05:10.900Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}