jackc/pgx · error

insufficient bytes

Error message

insufficient bytes

What it means

ErrInsufficientBytes is the sentinel (internal/pgio/read.go:12) wrapped by every bounds-check failure in the pgio.Reader — a fixed-size read (Byte/Uint16/…/Int64), Bytes(n), CString (unterminated), or Count (count exceeds remaining) tried to advance past the end of the source byte slice. It surfaces from a pgtype Codec decoding a PostgreSQL binary value whose wire bytes are shorter than the type's layout requires.

Source

Thrown at internal/pgio/read.go:12

package pgio

import (
	"bytes"
	"encoding/binary"
	"errors"
	"fmt"
)

// ErrInsufficientBytes is wrapped by all Reader errors caused by a read past
// the end of the source.
var ErrInsufficientBytes = errors.New("insufficient bytes")

// Reader is a bounds-checked reader for the PostgreSQL binary format. It is
// designed so that decoders of untrusted input cannot forget a length check:
// every read validates against the remaining bytes, and the first failure
// sticks. After a failure all subsequent reads return zero values, so a
// decoder can read an entire structure without intermediate error checks and
// inspect Err or Finish once at the end.
//
// Reads never panic. A decoder that branches on a value it just read (e.g. an
// element count used to size an allocation) should check Err before acting on
// the value.
type Reader struct {
	s   []byte
	rp  int
	err error
}

func NewReader(s []byte) *Reader {

View on GitHub (pinned to ec1a0befd2)

Solutions

1. If it reproduces on a specific column, capture the raw bytes and confirm the server is sending the documented binary format for that type OID.
2. Update pgx and any pgtype-related extensions to match the PostgreSQL server version (codec/server version skew is the most common cause).
3. For custom codecs, audit reads to match the documented layout; prefer pgio.Reader over manual slicing so bounds are checked.
4. Check the column's typname/typreceive to ensure you are using the right codec; register the correct type if a default fallback is mis-decoding.
5. Inspect network/proxy (PgBouncer in non-session-pooling modes, a truncating proxy) if the bytes arrive short.

Example fix

// before (custom codec over-reading)
func (c *MyCodec) DecodeBinary(m *pgtype.Map, src []byte) (any, error) {
    return binary.BigEndian.Uint32(src[:4]), nil // panics or wrong if src<4
}

// after — use pgio.Reader, surface ErrInsufficientBytes cleanly
func (c *MyCodec) DecodeBinary(m *pgtype.Map, src []byte) (any, error) {
    r := pgio.NewReader(src)
    v := r.Uint32()
    return v, r.Err()
}

func isInsufficientBytes(err error) bool { return errors.Is(err, pgio.ErrInsufficientBytes) }

v, err := codec.DecodeBinary(typeMap, src)
if errors.Is(err, pgio.ErrInsufficientBytes) {
    // truncated/corrupt payload; log OID + len(src) and drop/skip
    log.Printf("short binary payload for oid=%d len=%d", oid, len(src))
    return nil, err
}

When it happens

Trigger: Decoding a binary-format value whose payload is truncated: a corrupted/too-short binary column, a server/extension sending malformed data, a codec reading a field the message did not include, or a hand-rolled codec that over-reads. Also from CString on a non-NUL-terminated string.

Common situations: A column of a custom or extension type whose binary format pgx's codec parses incorrectly (version skew); a network/proxy truncating messages; a corrupted pg_dump/restore; an extension returning an unexpected payload shape; a hand-written Codec that mis-sizes reads.

Related errors

• invalid length

AI-assisted analysis of jackc/pgx@ec1a0befd2 (2026-08-04). Data as JSON: /data/errors/5b79d03c63bd8fa9.json. Report an issue: GitHub.