What causes "mlkem: invalid encoding length" in golang/go?

Decode base64/PEM and assert the total length equals the variant's expected public-key/ciphertext size before parsing.

mlkem: invalid encoding length

Error message

mlkem: invalid encoding length

What it means

ML-KEM (FIPS 203) encodes each polynomial as 384 bytes via ByteDecode/ByteEncode₁₂ (twelve bits per coefficient, q=3329, 256 coefficients). polyByteDecode checks that the input slice is exactly encodingSize12 bytes before reading; otherwise it returns 'invalid encoding length'. This is the first validation a public key or ciphertext polynomial undergoes during decapsulation.

Source

Thrown at src/crypto/internal/fips140/mlkem/field.go:166

	out, B := sliceForAppend(b, encodingSize12)
	for i := 0; i < n; i += 2 {
		x := uint32(f[i]) | uint32(f[i+1])<<12
		B[0] = uint8(x)
		B[1] = uint8(x >> 8)
		B[2] = uint8(x >> 16)
		B = B[3:]
	}
	return out
}

// polyByteDecode decodes the 384-byte encoding of a polynomial, checking that
// all the coefficients are properly reduced. This fulfills the "Modulus check"
// step of ML-KEM Encapsulation.
//
// It implements ByteDecode₁₂, according to FIPS 203, Algorithm 6.
func polyByteDecode[T ~[n]fieldElement](b []byte) (T, error) {
	if len(b) != encodingSize12 {
		return T{}, errors.New("mlkem: invalid encoding length")
	}
	var f T
	for i := 0; i < n; i += 2 {
		d := uint32(b[0]) | uint32(b[1])<<8 | uint32(b[2])<<16
		const mask12 = 0b1111_1111_1111
		var err error
		if f[i], err = fieldCheckReduced(uint16(d & mask12)); err != nil {
			return T{}, errors.New("mlkem: invalid polynomial encoding")
		}
		if f[i+1], err = fieldCheckReduced(uint16(d >> 12)); err != nil {
			return T{}, errors.New("mlkem: invalid polynomial encoding")
		}
		b = b[3:]
	}
	return f, nil
}

// sliceForAppend takes a slice and a requested number of bytes. It returns a

View on GitHub (pinned to b6b368adc5)

Solutions

Decode base64/PEM and assert the total length equals the variant's expected public-key/ciphertext size before parsing.
Slice outer blobs by the variant's polynomial counts (k=2/3/4) so each 384-byte block is whole.
Round-trip test serialization to catch off-by-one slicing in the parser.

Example fix

// before
dk, err := mlkem.NewDecapsulationKey1024(shortSeed)  // wrong API / wrong size

// after
b, _ := base64.StdEncoding.DecodeString(b64)
if len(b)%mlkem.EncodingSize12 != 0 { return ErrBadKey }
dk, err := mlkem.NewDecapsulationKey1024(b)

Defensive patterns

Strategy: validation

Validate before calling

if len(b)%mlkem.EncodingSize12 != 0 || len(b) == 0 {
    return ErrBadPolynomialEncoding
}

Type guard

func isWholePolynomialBlock(b []byte) bool { return len(b)%384 == 0 && len(b) > 0 }

Prevention

Decode base64/PEM and assert total length before parsing ML-KEM blobs.
Slice outer structures by the variant's polynomial count (k=2/3/4).
Round-trip test serialization to catch off-by-one slicing.

When it happens

Trigger: polyByteDecode is called (transitively, via NewDecapsulationKey*, Encapsulate, Decapsulate, or the testing-only NIST parser) with a slice whose length is not 384.

Common situations: Truncating a key/ciphertext blob; base64/PEM envelope not stripped; slicing the wrong number of polynomials out of an outer structure; using an ML-KEM-768/512 blob where ML-KEM-1024 expects more polynomials.

Related errors

AI-assisted analysis of golang/go@b6b368adc5 (2026-08-12). Data as JSON: /api/errors/ee8d4bce7b31d329. Report an issue: GitHub.