eclipse-vertx/vert.x · error · VertxException

Invalid PKCS8 encoding: not a sequence

Error message

Invalid PKCS8 encoding: not a sequence

What it means

PrivateKeyParser.getPKCS8EncodedKeyAlgorithm expects the encoded key to be a DER SEQUENCE at the top level (PKCS#8 PrivateKeyInfo). If the first ASN.1 object is not a SEQUENCE, it throws VertxException('Invalid PKCS8 encoding: not a sequence'), meaning the bytes are not a PKCS#8 structure.

Source

Thrown at vertx-core/src/main/java/io/vertx/core/net/impl/pkcs1/PrivateKeyParser.java:95

      throw new VertxException("Cannot determine EC parameter spec for curve name/OID", e);
    }
  }

  /**
   * Gets the algorithm used by a PKCS#8 encoded private key.
   *
   * @param encodedKey The encoded private key.
   * @return The algorithm name, either <em>RSA</em> or <em>EC</em>, depending on
   *         the algorithm identifier found in the encoded key.
   * @throws VertxException if the key is not PKCS#8 encoded or uses an unsupported
   *         algorithm.
   */
  public static String getPKCS8EncodedKeyAlgorithm(byte[] encodedKey) {

    DerParser parser = new DerParser(encodedKey);
    Asn1Object sequence = parser.read();
    if (sequence.getType() != DerParser.SEQUENCE) {
      throw new VertxException("Invalid PKCS8 encoding: not a sequence");
    }

    parser = sequence.getParser();
    BigInteger version = parser.read().getInteger();
    if (version.intValue() != 0) {
        throw new VertxException("Unsupported version, expected 0 but found " + version.intValue());
    }

    sequence = parser.read();
    if (sequence.getType() != DerParser.SEQUENCE) {
        throw new VertxException("Invalid PKCS8 encoding: could not read Algorithm Identifier");
    }

    parser = sequence.getParser();
    byte[] algorithmIdentifier = parser.read().getObjectIdentifier();
    if (Arrays.equals(OID_RSA_PUBLIC_KEY, algorithmIdentifier)) {
        return "RSA";
    } else if (Arrays.equals(OID_EC_PUBLIC_KEY, algorithmIdentifier)) {

View on GitHub (pinned to fb308bd8c3)

Solutions

  1. Convert the key to PKCS#8: openssl pkcs8 -topk8 -nocrypt -in key.pem -out key8.pem
  2. Ensure the PEM header is '-----BEGIN PRIVATE KEY-----' before calling this parser
  3. Strip encryption/DEK-Info first if the key is encrypted

Example fix

// before
// key.pem: -----BEGIN RSA PRIVATE KEY----- (PKCS#1)
String alg = PrivateKeyParser.getPKCS8EncodedKeyAlgorithm(der);
// after
// $ openssl pkcs8 -topk8 -nocrypt -in key.pem -out key8.pem
// key8.pem: -----BEGIN PRIVATE KEY----- (PKCS#8)
String alg = PrivateKeyParser.getPKCS8EncodedKeyAlgorithm(der);
Defensive patterns

Strategy: validation

Validate before calling

String pemHeader = Files.readString(pemPath).split("\n")[0];
if (!pemHeader.contains("BEGIN PRIVATE KEY")) {
  throw new IllegalArgumentException("expected PKCS#8 PEM ('BEGIN PRIVATE KEY'), got: " + pemHeader);
}

Try / catch

try {
  alg = PrivateKeyParser.getPKCS8EncodedKeyAlgorithm(der);
} catch (VertxException e) {
  // convert: openssl pkcs8 -topk8 -nocrypt -in key.pem -out key8.pem
}

Prevention

When it happens

Trigger: Passing a PKCS#1 ('BEGIN RSA PRIVATE KEY') key, a raw SEC1 EC key ('BEGIN EC PRIVATE KEY'), a public key, or an encrypted PEM body to getPKCS8EncodedKeyAlgorithm.

Common situations: Users hand a legacy 'RSA PRIVATE KEY' file where 'PRIVATE KEY' (PKCS#8) is expected; encrypted keys whose body is not plain DER; truncated/mis-decoded base64.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors


AI-assisted analysis of eclipse-vertx/vert.x@fb308bd8c3 (2026-09-06). Data as JSON: /api/errors/37b54d4802dba13b. Report an issue: GitHub.