XTLS/Xray-core · error

failed to decode key

Error message

failed to decode key

What it means

Returned by ParseCertificate when pem.Decode finds no PEM block in the private-key input. The key material must be a PEM block (e.g. RSA/EC PRIVATE KEY or PKCS#8); anything else fails here. Note the parser assumes RSA-style single-block keys (ToPEM re-encodes as 'RSA PRIVATE KEY').

Source

Thrown at common/protocol/tls/cert/cert.go:34

	"github.com/xtls/xray-core/common"
	"github.com/xtls/xray-core/common/errors"
)

type Certificate struct {
	// certificate in ASN.1 DER format
	Certificate []byte
	// Private key in ASN.1 DER format
	PrivateKey []byte
}

func ParseCertificate(certPEM []byte, keyPEM []byte) (*Certificate, error) {
	certBlock, _ := pem.Decode(certPEM)
	if certBlock == nil {
		return nil, errors.New("failed to decode certificate")
	}
	keyBlock, _ := pem.Decode(keyPEM)
	if keyBlock == nil {
		return nil, errors.New("failed to decode key")
	}
	return &Certificate{
		Certificate: certBlock.Bytes,
		PrivateKey:  keyBlock.Bytes,
	}, nil
}

func (c *Certificate) ToPEM() ([]byte, []byte) {
	return pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: c.Certificate}),
		pem.EncodeToMemory(&pem.Block{Type: "RSA PRIVATE KEY", Bytes: c.PrivateKey})
}

type Option func(*x509.Certificate)

func Authority(isCA bool) Option {
	return func(cert *x509.Certificate) {
		cert.IsCA = isCA
	}

View on GitHub (pinned to 7d214f8b09)

Solutions

1. Ensure the key is an unencrypted PEM private key: `openssl rsa -in key.pem -check -noout`
2. Decrypt encrypted keys first: `openssl rsa -in encrypted.pem -out plain.pem`
3. Convert DER keys: `openssl rsa -inform der -in key.der -out key.pem`

Example fix

# decrypt an encrypted key before use
openssl rsa -in encrypted_key.pem -out key.pem

block, _ := pem.Decode(keyPEM)
if block == nil || block.Type == "ENCRYPTED PRIVATE KEY" {
    return errors.New("key must be an unencrypted PEM private key")
}

func isPEMPrivateKey(b []byte) bool { block, _ := pem.Decode(b); return block != nil && strings.HasSuffix(block.Type, "PRIVATE KEY") }

When it happens

Trigger: Calling ParseCertificate with a DER-encoded key, an empty keyPEM, or a modern PKCS#8 'PRIVATE KEY' file can still decode as a PEM block - but encrypted ('ENCRYPTED PRIVATE KEY') or non-PEM input returns nil from pem.Decode and triggers this error.

Common situations: Using an encrypted private key without decrypting it first, key files in DER format, or an empty key file due to a bad deploy.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• no certificates were found while parsing the bundle
• failed to decode certificate
• failed to produce report
• invalid version component %s in %s
• varint overflow

AI-assisted analysis of XTLS/Xray-core@7d214f8b09 (2026-08-15). Data as JSON: /api/errors/f302bfd6d987c4d5. Report an issue: GitHub.