golang/go · error

mlkem: invalid polynomial encoding

Error message

mlkem: invalid polynomial encoding

What it means

After confirming the 384-byte length, polyByteDecode unpacks each pair of coefficients (12 bits each) and calls fieldCheckReduced to ensure every value is < q (3329). A value ≥ 3329 means the encoding is non-canonical, which FIPS 203 explicitly forbids during decapsulation (the 'Modulus check' step); 'invalid polynomial encoding' is returned. This rejects malformed keys/ciphertexts that could leak information through non-canonical encodings.

Source

Thrown at src/crypto/internal/fips140/mlkem/field.go:174

	return out
}

// polyByteDecode decodes the 384-byte encoding of a polynomial, checking that
// all the coefficients are properly reduced. This fulfills the "Modulus check"
// step of ML-KEM Encapsulation.
//
// It implements ByteDecode₁₂, according to FIPS 203, Algorithm 6.
func polyByteDecode[T ~[n]fieldElement](b []byte) (T, error) {
	if len(b) != encodingSize12 {
		return T{}, errors.New("mlkem: invalid encoding length")
	}
	var f T
	for i := 0; i < n; i += 2 {
		d := uint32(b[0]) | uint32(b[1])<<8 | uint32(b[2])<<16
		const mask12 = 0b1111_1111_1111
		var err error
		if f[i], err = fieldCheckReduced(uint16(d & mask12)); err != nil {
			return T{}, errors.New("mlkem: invalid polynomial encoding")
		}
		if f[i+1], err = fieldCheckReduced(uint16(d >> 12)); err != nil {
			return T{}, errors.New("mlkem: invalid polynomial encoding")
		}
		b = b[3:]
	}
	return f, nil
}

// sliceForAppend takes a slice and a requested number of bytes. It returns a
// slice with the contents of the given slice followed by that many bytes and a
// second slice that aliases into it and contains only the extra bytes. If the
// original slice has sufficient capacity then no allocation is performed.
func sliceForAppend(in []byte, n int) (head, tail []byte) {
	if total := len(in) + n; cap(in) >= total {
		head = in[:total]
	} else {
		head = make([]byte, total)

View on GitHub (pinned to b6b368adc5)

Solutions

1. Treat as an invalid ciphertext/key: return a decapsulation failure and, per FIPS 203, fall back to the implicit-rejection shared secret only if that is your protocol's policy.
2. Re-acquire the ciphertext from the peer; if reproducible, suspect a non-conformant peer implementation.
3. Add an integrity wrapper (AEAD/checksum) around stored ML-KEM blobs so corruption is caught before decapsulation.

Example fix

// before
ss, err := dk.DecapsulateClient(ct)  // ct has a non-canonical coefficient
if err != nil { panic(err) }

// after
ss, err := dk.DecapsulateClient(ct)
if err != nil { return ErrCiphertextInvalid }

if _, err := dk.DecapsulateClient(ct); err != nil {
    return ErrCiphertextInvalid // do not distinguish modulus-check sub-failures
}

When it happens

Trigger: A 384-byte polynomial block whose bit-packed 12-bit words contain a value ≥ 3329 (the even-indexed coefficient of any pair).

Common situations: Corrupted ciphertext; a peer implementation that does not range-reduce coefficients; tampering intended to probe the decapsulator; bit flips in stored key material.

Related errors

• mldsa: invalid signature
• mldsa: invalid signature encoding
• ecdsa: public key does not match curve
• ecdsa: invalid signature: r is zero
• ecdsa: invalid signature: s is zero

AI-assisted analysis of golang/go@b6b368adc5 (2026-08-12). Data as JSON: /api/errors/6afb982157785b88. Report an issue: GitHub.