apache/beam · error

invalid varuintz encoding for: %v

Error message

invalid varuintz encoding for: %v

What it means

decVarUintZ decodes bytes with binary.Uvarint. If Uvarint returns size <= 0 the data is malformed or truncated and this error is returned. Note this function is also the inner decode step for the float coder, so a float decode failure can surface from here first.

Source

Thrown at sdks/go/pkg/beam/core/runtime/coderx/varint.go:116

		val = uint64(n)
	case uint16:
		val = uint64(n)
	case uint32:
		val = uint64(n)
	case uint64:
		val = n
	default:
		panic(fmt.Sprintf("received unknown value type: want an unsigned integer:, got %T", n))
	}
	ret := make([]byte, binary.MaxVarintLen64)
	size := binary.PutUvarint(ret, val)
	return ret[:size]
}

func decVarUintZ(t reflect.Type, data []byte) (typex.T, error) {
	n, size := binary.Uvarint(data)
	if size <= 0 {
		return nil, errors.Errorf("invalid varuintz encoding for: %v", data)
	}
	switch t.Kind() {
	case reflect.Uint:
		return uint(n), nil
	case reflect.Uint8:
		return uint8(n), nil
	case reflect.Uint16:
		return uint16(n), nil
	case reflect.Uint32:
		return uint32(n), nil
	case reflect.Uint64:
		return n, nil
	default:
		panic(fmt.Sprintf("unreachable statement: expected an unsigned integer, got %v", t))
	}
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure the data was encoded with the matching encVarUintZ / encFloat scheme.
2. Fix record framing so the full varint bytes are present before decoding.
3. Verify you aren't feeding varintz (signed) data into the unsigned decoder.
4. Log the raw bytes to confirm the leading varint header is valid.

func validVarUintZ(data []byte) bool {
    _, size := binary.Uvarint(data)
    return size > 0
}

func isVarUintZCoder(c *coder.CustomCoder) bool { return c.Name == "varuintz" }

val, err := coder.Decode(data)
if err != nil {
    if strings.Contains(err.Error(), "invalid varuintz encoding") {
        log.Warnf("bad varuintz record: % x", data)
        return nil, nil // dead-letter instead
    }
    return nil, err
}

When it happens

Trigger: Decoding (via 'varuintz' custom coder or decFloat) byte slices not produced by encVarUintZ — truncated data, wrong coder, or corrupted payload.

Common situations: Mixing signed varintz and unsigned varuintz data; reading partial buffers from network streams; old data written under a different encoding.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• invalid varintz encoding for: %v
• invalid float encoding for: %v
• VarLong not terminated.
• ReadSimpleRowHeader reading nils count: %v, %v
• error decoding bool: %v

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9dd1aea663c4136b. Report an issue: GitHub.