{"record":{"id":"7ea1a0c5d0f2ffcf","repo":"t8y2/dbx","slug":"zookeeper-request-frame-is-d-bytes-maximum-is-d","errorCode":null,"errorMessage":"ZooKeeper request frame is %d bytes, maximum is %d","messagePattern":"ZooKeeper request frame is (.+?) bytes, maximum is (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"agents/drivers/argo-go/zookeeper_protocol.go","lineNumber":340,"sourceCode":"\tif xid != client.xid {\n\t\treturn nil, fmt.Errorf(\"ZooKeeper response XID %d does not match request XID %d\", xid, client.xid)\n\t}\n\tif _, err := decoder.int64(); err != nil {\n\t\treturn nil, err\n\t}\n\tcode, err := decoder.int32()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif err := zooKeeperError(code); err != nil {\n\t\treturn nil, err\n\t}\n\treturn decoder.remaining(), nil\n}\n\nfunc (client *protocolZooKeeperClient) writeFrame(payload []byte) error {\n\tif len(payload) > zooKeeperMaxFrameSize {\n\t\treturn fmt.Errorf(\"ZooKeeper request frame is %d bytes, maximum is %d\", len(payload), zooKeeperMaxFrameSize)\n\t}\n\tif err := client.connection.SetWriteDeadline(time.Now().Add(client.timeout)); err != nil {\n\t\treturn err\n\t}\n\theader := make([]byte, 4)\n\tbinary.BigEndian.PutUint32(header, uint32(len(payload)))\n\tif err := writeAll(client.connection, header); err != nil {\n\t\treturn err\n\t}\n\treturn writeAll(client.connection, payload)\n}\n\nfunc (client *protocolZooKeeperClient) readFrame() ([]byte, error) {\n\tif err := client.connection.SetReadDeadline(time.Now().Add(client.timeout)); err != nil {\n\t\treturn nil, err\n\t}\n\theader := make([]byte, 4)\n\tif _, err := io.ReadFull(client.connection, header); err != nil {","sourceCodeStart":322,"sourceCodeEnd":358,"githubUrl":"https://github.com/t8y2/dbx/blob/c0390bff16418b651f4728520d99adf8ce48829a/agents/drivers/argo-go/zookeeper_protocol.go#L322-L358","documentation":"The client refuses to send a ZooKeeper request whose serialized payload exceeds zooKeeperMaxFrameSize. The ZooKeeper wire protocol prefixes each request with a 4-byte big-endian length, so oversized payloads are rejected before writing to protect both the client and the server from invalid or hostile frames. This is a client-side pre-flight check in writeFrame, raised before any bytes hit the socket.","triggerScenarios":"Any request whose encoded payload (xid + type + path + data/acl/watch fields) exceeds zooKeeperMaxFrameSize: calling Create/Set/Data operations with very large node data, or setting a node with an ACL list so long the encoded frame exceeds the cap.","commonSituations":"Storing large blobs (images, serialized documents, big JSON) in znodes instead of object storage; a bug in the encoder producing a garbage-length buffer; writing more than the ~1MB default jute.maxbuffer the server itself enforces, where the client cap trips first.","solutions":["Shrink the payload: split large node data into chunked child znodes (e.g. chunk-N under a parent) and reassemble on read.","Move large payloads to object storage (S3/GCS) and store only a reference/URL in the znode.","Check what you are encoding — log len(payload) before the call and confirm the data size is what you expect (accidental double-encoding or embedding of a whole config file is common).","If you control deployment and genuinely need larger frames, raise zooKeeperMaxFrameSize in this driver together with the server's jute.maxbuffer; otherwise the server will reject the frame anyway."],"exampleFix":"// before\nerr := zkClient.Set(path, bigBlob, version) // bigBlob is 2 MB\n\n// after\nconst chunkSize = 512 * 1024\nfor i, chunk := range chunks(bigBlob, chunkSize) {\n    if err := zkClient.Create(fmt.Sprintf(\"%s/chunk-%d\", path, i), chunk, 0, zk.WorldACL(zk.PermAll)); err != nil {\n        return err\n    }\n}","handlingStrategy":"validation","validationCode":"const zooKeeperMaxFrameSize = 1 << 20 // match the driver's cap\nif len(payload) > zooKeeperMaxFrameSize {\n    return fmt.Errorf(\"payload too large for a single znode: %d bytes\", len(payload))\n}\n// otherwise proceed with the Set/Create call","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep znode payloads small (< ~512KB) by design; ZooKeeper is a coordination store, not a blob store.","Chunk large data across child znodes or store it externally with a reference in the znode.","Log payload sizes in dev/tests so oversized writes surface before production.","Remember the server also enforces jute.maxbuffer (~1MB); the client cap exists to fail fast before the server does."],"tags":["zookeeper","frame-size","payload-limit","protocol"],"backgroundTag":"payload-too-large","analyzedSha":"c0390bff16418b651f4728520d99adf8ce48829a","analyzedAt":"2026-09-05T23:05:10.900Z","contentChangedAt":"2026-09-05T23:05:10.900Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}