tailscale/tailscale · error

failed to parse pebble minica cert

Error message

failed to parse pebble minica cert

What it means

E2E-setup error returned when x509.CertPool.AppendCertsFromPEM(pebbleMiniCACert) returns false — the embedded pebbleMiniCACert constant could not be parsed as any PEM certificate. Since it is a compile-time constant in the e2e package, failure implies the constant itself is malformed rather than anything environment-dependent.

Source

Thrown at cmd/k8s-operator/e2e/setup.go:213

	)
	testCAs = x509.NewCertPool()
	if *fDevcontrol {
		// Deploy pebble and get its certs.
		if err = applyPebbleResources(ctx, kubeClient); err != nil {
			return 0, fmt.Errorf("failed to apply pebble resources: %w", err)
		}

		pebblePod, err := waitForPodReady(ctx, logger, kubeClient, ns, client.MatchingLabels{"app": "pebble"})
		if err != nil {
			return 0, fmt.Errorf("pebble pod not ready: %w", err)
		}

		if err = forwardLocalPortToPod(ctx, logger, restCfg, ns, pebblePod, 15000); err != nil {
			return 0, fmt.Errorf("failed to set up port forwarding to pebble: %w", err)
		}

		if ok := testCAs.AppendCertsFromPEM(pebbleMiniCACert); !ok {
			return 0, fmt.Errorf("failed to parse pebble minica cert")
		}

		var pebbleCAChain []byte
		for _, path := range []string{"/intermediates/0", "/roots/0"} {
			pem, err := pebbleGet(ctx, 15000, path)
			if err != nil {
				return 0, err
			}
			pebbleCAChain = append(pebbleCAChain, pem...)
		}

		if ok := testCAs.AppendCertsFromPEM(pebbleCAChain); !ok {
			return 0, fmt.Errorf("failed to parse pebble ca chain cert")
		}

		if err = os.MkdirAll(certsDir, 0755); err != nil {
			return 0, fmt.Errorf("failed to create certs dir: %w", err)
		}

View on GitHub (pinned to cfe32b8be6)

Solutions

1. Regenerate the constant from a valid PEM: openssl x509 -in mini-ca.pem -outform PEM, then embed the complete BEGIN..END block.
2. Sanity-check in Go: pem.Decode on the constant must return a *pem.Block of type CERTIFICATE.
3. If untouched upstream code hits this, report a bug with your Go version and any source patching (go:generate, lint --fix).

Example fix

// before: broken literal (truncated or missing PEM frame)
const pebbleMiniCACert = "-----BEGIN CERTIFICATE-----MIIB..." // no END line

// after: complete PEM block
const pebbleMiniCACert = `-----BEGIN CERTIFICATE-----
MIIB...
-----END CERTIFICATE-----
`

// Guard the constant once in a unit test so breakage is caught at build time
func TestPebbleMiniCACertIsValid(t *testing.T) {
	ok := x509.NewCertPool().AppendCertsFromPEM(pebbleMiniCACert)
	if !ok {
		t.Fatal("pebbleMiniCACert is not parseable PEM")
	}
}

func isValidPEMCert(pemBytes []byte) bool {
	block, _ := pem.Decode(pemBytes)
	if block == nil || block.Type != "CERTIFICATE" {
		return false
	}
	_, err := x509.ParseCertificate(block.Bytes)
	return err == nil
}

if ok := testCAs.AppendCertsFromPEM(pebbleMiniCACert); !ok {
	// constant is compile-time data: fail fast with a clear message
	return 0, fmt.Errorf("pebbleMiniCACert embedded in the e2e package is invalid; regenerate it from the pebble mini CA PEM")
}

When it happens

Trigger: AppendCertsFromPEM parses only CERTIFICATE PEM blocks; it returns false when the bytes are empty, truncated, have a bad PEM frame, or contain only other block types. With the shipped constant this should not occur; forks that regenerate or hand-edit pebbleMiniCACert (e.g. switching pebble versions with different test CA keys) can break it.

Common situations: Forking the e2e suite and pasting a new CA cert without the full BEGIN/END CERTIFICATE framing; accidental whitespace/escaping damage in the raw-string literal; build tooling mangling the source file.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• failed to apply pebble resources: %w
• pebble pod not ready: %w
• failed to parse pebble ca chain cert
• unexpected output: no delimiter
• unexpected output: key in cert

AI-assisted analysis of tailscale/tailscale@cfe32b8be6 (2026-08-15). Data as JSON: /api/errors/de58163916534efd. Report an issue: GitHub.