basecamp/fizzy · error · ActionPack::WebAuthn::InvalidKeyError

Invalid EC2 coordinate length

Error message

Invalid EC2 coordinate length

What it means

For P-256, both the x and y coordinates must be exactly 32 bytes (P256_COORDINATE_LENGTH); an ASN.1 EC point is 0x04 || x || y with fixed-width fields. If either coordinate has a different byte size, InvalidKeyError 'Invalid EC2 coordinate length' is raised before OpenSSL sees the key.

Solutions

1. Check both coordinates' bytesize before conversion and pad with zero bytes on the left ("\x00" * (32 - x.bytesize) + x) only when you control and trust the encoding pipeline.
2. If the sizes are 48/66 bytes, the credential is really P-384/P-521 — reject it and fix registration options instead of padding.
3. Keep COSE keys as binary (CBOR) end-to-end; never round-trip byte strings through JSON numbers or hex without re-padding.
4. Rescue InvalidKeyError and fail registration — a wrong-length coordinate cannot be verified.

Example fix

# before
key = cose_key.to_openssl_key # raises: x is 31 bytes after a JSON round-trip stripped \x00

# after — normalize coordinate width before conversion (trusted pipelines only)
x = cose_key.parameters[CoseKey::EC2_X_LABEL].rjust(32, "\x00")
y = cose_key.parameters[CoseKey::EC2_Y_LABEL].rjust(32, "\x00")
normalized = ActionPack::WebAuthn::CoseKey.new(key_type: 2, algorithm: -7, parameters: { -1 => 1, -2 => x, -3 => y })
key = normalized.to_openssl_key

x = cose_key.parameters[ActionPack::WebAuthn::CoseKey::EC2_X_LABEL]
y = cose_key.parameters[ActionPack::WebAuthn::CoseKey::EC2_Y_LABEL]
return render(json: { error: 'credential coordinates have wrong width' }, status: :bad_request) unless x.bytesize == 32 && y.bytesize == 32

def valid_p256_coordinate_width?(cose_key)
  x = cose_key.parameters[ActionPack::WebAuthn::CoseKey::EC2_X_LABEL]
  y = cose_key.parameters[ActionPack::WebAuthn::CoseKey::EC2_Y_LABEL]
  x.bytesize == 32 && y.bytesize == 32
end

begin
  key = cose_key.to_openssl_key
rescue ActionPack::WebAuthn::InvalidKeyError => e
  render json: { error: 'credential key invalid' }, status: :bad_request
end

When it happens

Trigger: Coordinates shorter than 32 bytes because leading zero bytes were stripped by a minimally-encoding relay (JSON integer round-trip, some CBOR encoders), or 48-byte coordinates from a P-384 key mislabeled as P-256.

Common situations: COSE maps transported through systems that treat byte strings as integers or bigints and normalize leading zeros; keys generated by non-conformant authenticators or test tools; coordinates hex-decoded to 31 bytes because the hex string lost a leading 00.

Related errors

• Invalid EC2 key: #
• Unsupported EC curve: #
• Invalid base64 encoding in authenticator data
• Invalid OKP key: #
• Invalid RSA key: #

AI-assisted analysis of basecamp/fizzy@7aabe74580 (2026-08-21). Data as JSON: /api/errors/aea8fc55a9b48062. Report an issue: GitHub.

Appendix: source

Thrown at lib/action_pack/web_authn/cose_key.rb:120

  # is not supported.
  def to_openssl_key
    case [ key_type, algorithm ]
    when [ EC2, ES256 ] then build_ec2_es256_key
    when [ OKP, EDDSA ] then build_okp_eddsa_key
    when [ RSA, RS256 ] then build_rsa_rs256_key
    else raise ActionPack::WebAuthn::UnsupportedKeyTypeError, "Unsupported COSE key type/algorithm: #{key_type}/#{algorithm}"
    end
  end

  private
    def build_ec2_es256_key
      curve = parameters[EC2_CURVE_LABEL]
      raise ActionPack::WebAuthn::UnsupportedKeyTypeError, "Unsupported EC curve: #{curve}" unless curve == P256

      x = parameters[EC2_X_LABEL]
      y = parameters[EC2_Y_LABEL]
      raise ActionPack::WebAuthn::InvalidKeyError, "Missing EC2 key coordinates" if x.nil? || y.nil?
      raise ActionPack::WebAuthn::InvalidKeyError, "Invalid EC2 coordinate length" unless x.bytesize == P256_COORDINATE_LENGTH && y.bytesize == P256_COORDINATE_LENGTH

      # Uncompressed point format: 0x04 || x || y
      public_key_bytes = [ UNCOMPRESSED_POINT_MARKER, *x.bytes, *y.bytes ].pack("C*")

      asn1 = OpenSSL::ASN1::Sequence([
        OpenSSL::ASN1::Sequence([
          OpenSSL::ASN1::ObjectId("id-ecPublicKey"),
          OpenSSL::ASN1::ObjectId("prime256v1")
        ]),
        OpenSSL::ASN1::BitString(public_key_bytes)
      ])

      OpenSSL::PKey::EC.new(asn1.to_der)
    rescue OpenSSL::PKey::PKeyError => error
      raise ActionPack::WebAuthn::InvalidKeyError, "Invalid EC2 key: #{error.message}"
    end

    def build_okp_eddsa_key

View on GitHub (pinned to 7aabe74580)