shwenzhang/AndResGuard · error · InvalidKeySpecException

Failed to load PKCS #8 encoded private key from

Error message

Failed to load PKCS #8 encoded private key from <keyFile>

What it means

apksigner failed to reconstruct a PrivateKey object from the PKCS #8 encoded key specification read from the --key file. The bytes were read (or decrypted) but the underlying JCA KeyFactory rejected them as a valid PKCS #8 private key, so signing cannot proceed. This is a rethrow wrapping the original InvalidKeySpecException, which is attached as the cause.

Solutions

1. Convert the key to unencrypted PKCS #8 DER: `openssl pkcs8 -topk8 -nocrypt -in key.pem -out key.p8` (add -outform DER if needed).
2. Check the wrapped cause (e.getCause()) for the exact KeyFactory rejection reason.
3. Verify the file actually is the private key, not the certificate, and that it is not truncated.
4. If the key is PEM (base64), strip the BEGIN/END headers and base64-decode, or re-export in DER form.

Example fix

// before
openssl genrsa -out key.pem 2048
apksigner sign --key key.pem --cert cert.pem --out app.apk app-unsigned.apk
// after
openssl genrsa -out key.pem 2048
openssl pkcs8 -topk8 -nocrypt -in key.pem -out key.p8
apksigner sign --key key.p8 --cert cert.pem --out app.apk app-unsigned.apk

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Base64;

static boolean isPkcs8Der(byte[] der) {
  // PKCS #8 PrivateKeyInfo starts with SEQUENCE (0x30); byte 1 is version INTEGER 0
  return der.length > 4 && der[0] == 0x30 && der[1] > 0 && der[2] == 0x02 && der[3] == 0x01 && der[4] == 0x00;
}

static void validateKeyFile(String path) throws IOException {
  byte[] raw = Files.readAllBytes(Paths.get(path));
  String s = new String(raw).trim();
  if (s.startsWith("-----BEGIN")) throw new IllegalArgumentException(
      path + " is PEM; convert to PKCS#8 DER: openssl pkcs8 -topk8 -nocrypt -in " + path);
  if (s.contains("ENCRYPTED")) throw new IllegalArgumentException(path + " is encrypted; remove passphrase");
  byte[] der = s.startsWith("-----") ? raw : Base64.getDecoder().decode(s.replaceAll("\\s", ""));
  if (!isPkcs8Der(der)) throw new IllegalArgumentException(
      path + " is not PKCS#8 DER; run: openssl pkcs8 -topk8 -nocrypt -in key.pem -out key.p8");
}

try {
  sign(params);
} catch (InvalidKeySpecException e) {
  System.err.println("Key file format rejected: " + e.getMessage()
      + "; cause=" + e.getCause()
      + ". Convert with: openssl pkcs8 -topk8 -nocrypt -in key.pem -out key.p8");
  throw e;
}

When it happens

Trigger: Running `apksigner sign --key <file> ...` where the file's contents are not a valid unencrypted PKCS #8 DER key: e.g. the file holds a PKCS #1 (RSA) key, an encrypted PKCS #8 key with a wrong/missing password path, a PEM-encoded key, or a truncated/corrupted key file.

Common situations: Exporting a key from OpenSSL in the wrong format (`openssl genrsa` produces PKCS #1, not PKCS #8); forgetting to convert with `openssl pkcs8 -topk8`; supplying a certificate file to --key by mistake; a key corrupted during transfer (CRLF mangling, truncation).

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• Not an RSA, EC, or DSA private key
• Failed to parse encrypted private key blob
• No keystore passwords
• No key passwords
• No passwords

AI-assisted analysis of shwenzhang/AndResGuard@e4df245d82 (2026-09-12). Data as JSON: /api/errors/eb5ebfc53c4a0c34. Report an issue: GitHub.

Appendix: source

Thrown at AndResGuard-core/src/main/java/apksigner/ApkSignerTool.java:810

        String passwordSpec = (keyPasswordSpec != null) ? keyPasswordSpec : PasswordRetriever.SPEC_STDIN;
        List<char[]> keyPasswords = passwordRetriver.getPasswords(passwordSpec, "Private key password for " + name);
        keySpec = decryptPkcs8EncodedKey(encryptedPrivateKeyInfo, keyPasswords);
      } catch (IOException e) {
        // The blob is not an encrypted private key blob
        if (keyPasswordSpec == null) {
          // Given that no password was specified, assume the blob is an unencrypted
          // private key blob
          keySpec = new PKCS8EncodedKeySpec(privateKeyBlob);
        } else {
          throw new InvalidKeySpecException("Failed to parse encrypted private key blob " + keyFile, e);
        }
      }

      // Load the private key from its PKCS #8 encoded form.
      try {
        privateKey = loadPkcs8EncodedPrivateKey(keySpec);
      } catch (InvalidKeySpecException e) {
        throw new InvalidKeySpecException("Failed to load PKCS #8 encoded private key from " + keyFile, e);
      }

      // Load certificates
      Collection<? extends Certificate> certs;
      try (FileInputStream in = new FileInputStream(certFile)) {
        certs = CertificateFactory.getInstance("X.509").generateCertificates(in);
      }
      List<X509Certificate> certList = new ArrayList<>(certs.size());
      for (Certificate cert : certs) {
        certList.add((X509Certificate) cert);
      }
      this.certs = certList;
    }
  }

  /**
   * Indicates that there is an issue with command-line parameters provided to this tool.
   */

View on GitHub (pinned to e4df245d82)