basecamp/fizzy · error · ActionPack::WebAuthn::InvalidKeyError

Invalid EC2 coordinate length

Error message

Invalid EC2 coordinate length

What it means

For P-256, both the x and y coordinates must be exactly 32 bytes (P256_COORDINATE_LENGTH); an ASN.1 EC point is 0x04 || x || y with fixed-width fields. If either coordinate has a different byte size, InvalidKeyError 'Invalid EC2 coordinate length' is raised before OpenSSL sees the key.

Source

Thrown at lib/action_pack/web_authn/cose_key.rb:120

  # is not supported.
  def to_openssl_key
    case [ key_type, algorithm ]
    when [ EC2, ES256 ] then build_ec2_es256_key
    when [ OKP, EDDSA ] then build_okp_eddsa_key
    when [ RSA, RS256 ] then build_rsa_rs256_key
    else raise ActionPack::WebAuthn::UnsupportedKeyTypeError, "Unsupported COSE key type/algorithm: #{key_type}/#{algorithm}"
    end
  end

  private
    def build_ec2_es256_key
      curve = parameters[EC2_CURVE_LABEL]
      raise ActionPack::WebAuthn::UnsupportedKeyTypeError, "Unsupported EC curve: #{curve}" unless curve == P256

      x = parameters[EC2_X_LABEL]
      y = parameters[EC2_Y_LABEL]
      raise ActionPack::WebAuthn::InvalidKeyError, "Missing EC2 key coordinates" if x.nil? || y.nil?
      raise ActionPack::WebAuthn::InvalidKeyError, "Invalid EC2 coordinate length" unless x.bytesize == P256_COORDINATE_LENGTH && y.bytesize == P256_COORDINATE_LENGTH

      # Uncompressed point format: 0x04 || x || y
      public_key_bytes = [ UNCOMPRESSED_POINT_MARKER, *x.bytes, *y.bytes ].pack("C*")

      asn1 = OpenSSL::ASN1::Sequence([
        OpenSSL::ASN1::Sequence([
          OpenSSL::ASN1::ObjectId("id-ecPublicKey"),
          OpenSSL::ASN1::ObjectId("prime256v1")
        ]),
        OpenSSL::ASN1::BitString(public_key_bytes)
      ])

      OpenSSL::PKey::EC.new(asn1.to_der)
    rescue OpenSSL::PKey::PKeyError => error
      raise ActionPack::WebAuthn::InvalidKeyError, "Invalid EC2 key: #{error.message}"
    end

    def build_okp_eddsa_key

View on GitHub (pinned to 7aabe74580)

Solutions

1. Check both coordinates' bytesize before conversion and pad with zero bytes on the left ("\x00" * (32 - x.bytesize) + x) only when you control and trust the encoding pipeline.
2. If the sizes are 48/66 bytes, the credential is really P-384/P-521 — reject it and fix registration options instead of padding.
3. Keep COSE keys as binary (CBOR) end-to-end; never round-trip byte strings through JSON numbers or hex without re-padding.
4. Rescue InvalidKeyError and fail registration — a wrong-length coordinate cannot be verified.

Example fix

# before
key = cose_key.to_openssl_key # raises: x is 31 bytes after a JSON round-trip stripped \x00

# after — normalize coordinate width before conversion (trusted pipelines only)
x = cose_key.parameters[CoseKey::EC2_X_LABEL].rjust(32, "\x00")
y = cose_key.parameters[CoseKey::EC2_Y_LABEL].rjust(32, "\x00")
normalized = ActionPack::WebAuthn::CoseKey.new(key_type: 2, algorithm: -7, parameters: { -1 => 1, -2 => x, -3 => y })
key = normalized.to_openssl_key

x = cose_key.parameters[ActionPack::WebAuthn::CoseKey::EC2_X_LABEL]
y = cose_key.parameters[ActionPack::WebAuthn::CoseKey::EC2_Y_LABEL]
return render(json: { error: 'credential coordinates have wrong width' }, status: :bad_request) unless x.bytesize == 32 && y.bytesize == 32

def valid_p256_coordinate_width?(cose_key)
  x = cose_key.parameters[ActionPack::WebAuthn::CoseKey::EC2_X_LABEL]
  y = cose_key.parameters[ActionPack::WebAuthn::CoseKey::EC2_Y_LABEL]
  x.bytesize == 32 && y.bytesize == 32
end

begin
  key = cose_key.to_openssl_key
rescue ActionPack::WebAuthn::InvalidKeyError => e
  render json: { error: 'credential key invalid' }, status: :bad_request
end

When it happens

Trigger: Coordinates shorter than 32 bytes because leading zero bytes were stripped by a minimally-encoding relay (JSON integer round-trip, some CBOR encoders), or 48-byte coordinates from a P-384 key mislabeled as P-256.

Common situations: COSE maps transported through systems that treat byte strings as integers or bigints and normalize leading zeros; keys generated by non-conformant authenticators or test tools; coordinates hex-decoded to 31 bytes because the hex string lost a leading 00.

Related errors

• Unsupported EC curve: #{curve}
• Invalid EC2 key: #{error.message}
• Invalid base64 encoding in authenticator data
• Unsupported COSE key type/algorithm: #{key_type}/#{algorithm
• Missing EC2 key coordinates

AI-assisted analysis of basecamp/fizzy@7aabe74580 (2026-08-21). Data as JSON: /api/errors/aea8fc55a9b48062. Report an issue: GitHub.