What causes "invalid P256 compressed point encoding" in golang/go?

Reject the peer's public key and request a valid one

invalid P256 compressed point encoding

Error message

invalid P256 compressed point encoding

What it means

When decompressing a P-256 point, the library computes y² = x³ - 3x + b and attempts to extract a square root via p256Sqrt. If p256Sqrt returns false, the value is not a quadratic residue mod p, meaning no valid y exists for the given x and the point is not on the curve. The encoding is therefore invalid.

Source

Thrown at src/crypto/internal/fips140/nistec/p256_asm.go:119

		if err := p256CheckOnCurve(&r.x, &r.y); err != nil {
			return nil, err
		}
		r.z = p256One
		return p.Set(&r), nil

	// Compressed form.
	case len(b) == p256CompressedLength && (b[0] == 2 || b[0] == 3):
		var r P256Point
		p256BigToLittle(&r.x, (*[32]byte)(b[1:33]))
		if p256LessThanP(&r.x) == 0 {
			return nil, errors.New("invalid P256 element encoding")
		}
		p256Mul(&r.x, &r.x, &rr)

		// y² = x³ - 3x + b
		p256Polynomial(&r.y, &r.x)
		if !p256Sqrt(&r.y, &r.y) {
			return nil, errors.New("invalid P256 compressed point encoding")
		}

		// Select the positive or negative root, as indicated by the least
		// significant bit, based on the encoding type byte.
		yy := new(p256Element)
		p256FromMont(yy, &r.y)
		cond := int(yy[0]&1) ^ int(b[0]&1)
		p256NegCond(&r.y, cond)

		r.z = p256One
		return p.Set(&r), nil

	default:
		return nil, errors.New("invalid P256 point encoding")
	}
}

// p256Polynomial sets y2 to x³ - 3x + b, and returns y2.

View on GitHub (pinned to b6b368adc5)

Solutions

Reject the peer's public key and request a valid one
Verify the key was generated for the correct curve (P-256/secp256r1/prime256v1)
Use crypto/ecdsa.UnmarshalCompressed or ecdh.P256().NewPublicKey which propagate this error for proper handling

Defensive patterns

Strategy: try-catch

Try / catch

_, err := point.SetBytes(b)
if err != nil {
    // The compressed x-coordinate does not yield a valid curve point.
    // This key is invalid or forged; do not use it.
    return fmt.Errorf("invalid compressed point (no valid y): %w", err)
}

Prevention

Reject peer public keys that fail decompression — they may be adversarial
Verify the curve parameter (P-256) matches what the peer claims
Always use the library's SetBytes which performs full validation rather than manual decompression

When it happens

Trigger: Calling P256Point.SetBytes with a compressed point whose x coordinate does not correspond to any actual point on the P-256 curve — i.e., x³ - 3x + b is a quadratic non-residue mod p.

Common situations: Random or fabricated data passed as a compressed public key; an x coordinate from a different curve (e.g., secp256k1) mistakenly used with P-256; bit-flip corruption in the x field.

Related errors

AI-assisted analysis of golang/go@b6b368adc5 (2026-08-12). Data as JSON: /api/errors/05600410e2ab0f90. Report an issue: GitHub.