jenkinsci/jenkins · error · IllegalArgumentException

Unknown public key type:

Error message

Unknown public key type: 

What it means

`Connection.detectKeyAlgorithm(PublicKey)` only recognizes RSA (`RSAPublicKey`) and DSA (`DSAPublicKey`) and throws IllegalArgumentException for any other key type. It is used by `proveIdentity`/`verifyIdentity` during CLI public-key authentication to pick the `SHA1with<algorithm>` Signature. Passing an EC, EdDSA, or other key pair is unsupported by this code path.

Source

Thrown at core/src/main/java/hudson/cli/Connection.java:237

     *
     * Cryptographic utility code.
     */
    public static byte[] fold(byte[] bytes, int size) {
        byte[] r = new byte[size];
        for (int i = Math.max(bytes.length, size) - 1; i >= 0; i--) {
            r[i % r.length] ^= bytes[i % bytes.length];
        }
        return r;
    }

    private String detectKeyAlgorithm(KeyPair kp) {
        return detectKeyAlgorithm(kp.getPublic());
    }

    private String detectKeyAlgorithm(PublicKey kp) {
        if (kp instanceof RSAPublicKey)     return "RSA";
        if (kp instanceof DSAPublicKey)     return "DSA";
        throw new IllegalArgumentException("Unknown public key type: " + kp);
    }

    /**
     * Used in conjunction with {@link #verifyIdentity(byte[])} to prove
     * that we actually own the private key of the given key pair.
     */
    public void proveIdentity(byte[] sharedSecret, KeyPair key) throws IOException, GeneralSecurityException {
        String algorithm = detectKeyAlgorithm(key);
        writeUTF(algorithm);
        writeKey(key.getPublic());

        Signature sig = Signature.getInstance("SHA1with" + algorithm);
        sig.initSign(key.getPrivate());
        sig.update(key.getPublic().getEncoded());
        sig.update(sharedSecret);
        writeObject(sig.sign());
    }

View on GitHub (pinned to 2e228ff40b)

Solutions

1. Generate and register an RSA key pair for Jenkins CLI authentication: `ssh-keygen -t rsa -b 4096` and add the public key to your Jenkins user.
2. If you must use the existing key, confirm its type; convert/regenerate as RSA or DSA since detectKeyAlgorithm supports only those.
3. Patch detectKeyAlgorithm (if you maintain a fork) to handle EC/EdDSA and the corresponding Signature algorithms.

Example fix

// before: Ed25519/EC key -> 'Unknown public key type'
//   ssh-keygen -t ed25519 -f ~/.ssh/jenkins_ed25519
//   java -jar jenkins-cli.jar -ssh -user me -i ~/.ssh/jenkins_ed25519 help
//
// after: use RSA which detectKeyAlgorithm accepts
//   ssh-keygen -t rsa -b 4096 -f ~/.ssh/jenkins_rsa
//   # add ~/.ssh/jenkins_rsa.pub to Jenkins user > Configure > SSH Public Keys
//   java -jar jenkins-cli.jar -ssh -user me -i ~/.ssh/jenkins_rsa help

// Only RSA/DSA keys are supported by Connection.detectKeyAlgorithm:
PublicKey pub = keyPair.getPublic();
if (!(pub instanceof RSAPublicKey) && !(pub instanceof DSAPublicKey)) {
    throw new IllegalArgumentException(
        "Jenkins CLI auth requires an RSA or DSA key, got: " + pub.getAlgorithm());
}
// proceed to proveIdentity(...)

// Type guard narrowing to supported key types
public static boolean isSupportedCliKey(KeyPair kp) {
    PublicKey p = kp.getPublic();
    return p instanceof RSAPublicKey || p instanceof DSAPublicKey;
}

When it happens

Trigger: Configuring Jenkins CLI SSH/public-key auth with a key pair whose public key is neither RSA nor DSA (e.g. an ECDSA `ECParameterSpec` key, or an Ed25519 key from newer OpenSSH), then connecting in a way that triggers proveIdentity. The algorithm detection fails before signing.

Common situations: Modern OpenSSH defaults generating Ed25519 keys; users copy those into Jenkins CLI auth; or a Java keypair generator defaulting to EC. RSA/DSA were the historically supported types.

Related errors

• DataInputStream unexpectedly returned negative integer
• Unsupported private key file format: key
• Multiple private key pairs N/A: key
• Cannot build {0} because its configuration has not been save
• Build scheduling Refused by an extension, hence not in Queue

AI-assisted analysis of jenkinsci/jenkins@2e228ff40b (2026-08-14). Data as JSON: /api/errors/1b57bd85aca72c32. Report an issue: GitHub.