What causes "invalid ZooKeeper buffer length %d" in t8y2/dbx?

Close and reopen the connection — once desynced, subsequent decodes will keep producing garbage.

invalid ZooKeeper buffer length %d

Error message

invalid ZooKeeper buffer length %d

What it means

The ZooKeeper binary decoder's bytes() reads an int32 length prefix for a byte buffer. Per the protocol, -1 means a null buffer (returned as nil); any other negative length is invalid and triggers this error. It guards against corrupted streams where a bogus negative value would otherwise be used as an allocation size.

Source

Thrown at agents/drivers/hive-go/zookeeper_protocol.go:491

func (decoder *zooKeeperDecoder) int64() (int64, error) {
	value, err := decoder.take(8)
	if err != nil {
		return 0, err
	}
	return int64(binary.BigEndian.Uint64(value)), nil
}

func (decoder *zooKeeperDecoder) bytes() ([]byte, error) {
	length, err := decoder.int32()
	if err != nil {
		return nil, err
	}
	if length == -1 {
		return nil, nil
	}
	if length < -1 {
		return nil, fmt.Errorf("invalid ZooKeeper buffer length %d", length)
	}
	value, err := decoder.take(int(length))
	if err != nil {
		return nil, err
	}
	return append([]byte(nil), value...), nil
}

func (decoder *zooKeeperDecoder) string() (string, error) {
	value, err := decoder.bytes()
	return string(value), err
}

func (decoder *zooKeeperDecoder) strings() ([]string, error) {
	length, err := decoder.int32()
	if err != nil {
		return nil, err
	}

View on GitHub (pinned to c0390bff16)

Solutions

Close and reopen the connection — once desynced, subsequent decodes will keep producing garbage.
Verify TLS is configured identically on client and server if the server requires it; a plaintext-vs-TLS mismatch garbles every frame.
Reproduce with a raw read (or readZooKeeperTestFrame) and hexdump the response to confirm the length prefix value and stream alignment.
Check the writer of the offending znode's data for corruption or a buggy custom serializer.
Upgrade the client/server pair to compatible ZooKeeper versions to rule out protocol incompatibility.

Example fix

// before
value, err := zkClient.Get("/cfg/node") // previous failed read left stream desynced
// after
client, err := hive.Dial(connStr) // reconnect on any decode error
value, err := client.Get("/cfg/node")

Defensive patterns

Strategy: retry

Validate before calling

// ensure the data you wrote is well-formed before reading it back
if len(payload) == 0 {
	return errors.New("refusing to write empty payload to " + path)
}
if _, err := client.Set(path, payload, -1); err != nil {
	return fmt.Errorf("write znode: %w", err)
}

Type guard

func isInvalidBufferLength(err error) bool {
	return err != nil && strings.Contains(err.Error(), "invalid ZooKeeper buffer length")
}

Try / catch

value, err := client.Get(path)
if err != nil {
	if isInvalidBufferLength(err) {
		// stream is desynced: rebuild the connection once, then retry
		client.Close()
		client, err = hive.Dial(connStr)
		if err == nil {
			value, err = client.Get(path)
		}
	}
	if err != nil {
		return fmt.Errorf("zk get %s: %w", path, err)
	}
}

Prevention

Only write znode data through conforming clients; avoid custom serializers for stored payloads.
Treat any decode error as fatal for the connection and immediately reconnect.
Match TLS settings exactly on client and server to prevent garbled frames.
Validate data round-trips in CI by writing and reading back via the same library.
Monitor for this error as an early signal of a corrupted or truncated network path.

When it happens

Trigger: Called from Get (getData), authenticateSASL, and the string()/test-serving paths whenever a decoded buffer length prefix is < -1 — i.e. the byte stream is misaligned or corrupted at that position.

Common situations: Reading a znode whose stored data is corrupted or was written by a non-conforming client, protocol desync after an earlier partial/failed read, a man-in-the-middle or truncated TLS stream, or pointing the client at something that isn't speaking the ZooKeeper binary protocol.

Related errors

AI-assisted analysis of t8y2/dbx@c0390bff16 (2026-09-05). Data as JSON: /api/errors/8ab63bcc8ad26be4. Report an issue: GitHub.