hyperledger/fabric · error

error decoding varint with proto.Buffer

Error message

error decoding varint with proto.Buffer

What it means

The buffer type in blkstorage wraps protobuf wire decoding. DecodeVarint consumes a varint from the remaining bytes using protowire.ConsumeVarint; if the bytes are not a valid varint (malformed encoding, >64 bits, or truncated), the parse error is wrapped as "error decoding varint with proto.Buffer". Callers use it to walk protobuf-serialized block fields directly.

Source

Thrown at common/ledger/blkstorage/protobuf_util.go:30

)

// buffer provides a wrapper on top of proto.Buffer.
// The purpose of this wrapper is to get to know the current position in the []byte
type buffer struct {
	buf      []byte
	position int
}

// newBuffer constructs a new instance of Buffer
func newBuffer(b []byte) *buffer {
	return &buffer{b, 0}
}

// DecodeVarint wraps the actual method and updates the position
func (b *buffer) DecodeVarint() (uint64, error) {
	v, n := protowire.ConsumeVarint(b.buf[b.position:])
	if n < 0 {
		return 0, errors.Wrap(protowire.ParseError(n), "error decoding varint with proto.Buffer")
	}
	b.position += n
	return v, nil
}

// DecodeRawBytes wraps the actual method and updates the position
func (b *buffer) DecodeRawBytes(alloc bool) ([]byte, error) {
	v, n := protowire.ConsumeBytes(b.buf[b.position:])
	if n < 0 {
		return nil, errors.Wrap(protowire.ParseError(n), "error decoding raw bytes with proto.Buffer")
	}
	b.position += n
	if alloc {
		v = append([]byte(nil), v...)
	}
	return v, nil
}

View on GitHub (pinned to 2736b63f8f)

Solutions

1. Identify which block/file offset produced the bad bytes (the extract helper's caller) and check the block file for truncation; roll back or re-fetch the ledger from peers.
2. Restore the ledger directory from a valid snapshot/backup or rebuild by resetting the ledger and resyncing from orderer/peers.
3. If constructing bytes yourself, verify the protobuf wire format is valid (complete varint, field key) before calling DecodeVarint.

if len(buf) == 0 {
    return errors.New("empty block bytes")
}
if _, n := protowire.ConsumeVarint(buf); n < 0 {
    return fmt.Errorf("not valid protobuf wire data: %v", protowire.ParseError(n))
}

v, err := b.DecodeVarint()
if err != nil {
    var perr error
    if errors.As(err, &perr) || errors.Is(err, os.ErrInvalid) {
        return fmt.Errorf("corrupt block record: %w", err)
    }
    return err
}

When it happens

Trigger: extractHeader, extractData, extractMetadata, or TestBuffer invoked on corrupted/truncated bytes: a partial block record in the block file (crash during append), a manually edited file, zero-length or empty envelope, or bytes whose leading varint field key is malformed.

Common situations: Reading a block file damaged by an unclean shutdown or disk corruption, restoring a ledger file from an incomplete backup, index/segment files out of sync with block files, or feeding non-protobuf bytes into the extract helpers.

Related errors

• error decoding raw bytes with proto.Buffer
• error unmarshalling last persisted commit hash
• error decoding the block number
• error decoding the data hash
• error decoding the previous hash

AI-assisted analysis of hyperledger/fabric@2736b63f8f (2026-09-04). Data as JSON: /api/errors/b6932f2da25fa005. Report an issue: GitHub.