intermediate certificate fetch recently failed for signer is

Error message

intermediate certificate fetch recently failed for signer issuer %q (cached)

What it means

intermediateCertPoolWithCaches maintains a negative TTL cache of failed intermediate-certificate fetches keyed by signer issuer. If a previous fetch for this issuer recently failed, the negative entry short-circuits the lookup and this error is returned instead of hammering the AIA endpoint again. The negative entry expires after its own (shorter) TTL, after which a new fetch is attempted.

Source

Thrown at upup/pkg/fi/cloudup/azure/attest.go:349

// intermediateCertPoolWithCaches performs a cached lookup against the supplied positive and
// negative TTL caches, invoking fetch on a miss. Tests inject their own stores and fetchers.
func intermediateCertPoolWithCaches(signer *x509.Certificate, fetch func(*x509.Certificate) (*x509.CertPool, error), positive, negative expirationcache.Store) (*x509.CertPool, error) {
	if signer == nil {
		return nil, fmt.Errorf("signer certificate is required")
	}

	keyStr := intermediateCacheKeyForSigner(signer)

	// Positive cache wins over negative: a successful later fetch overwrites any stale negative entry,
	// which expires on its own shorter TTL.
	if obj, ok, _ := positive.GetByKey(keyStr); ok {
		klog.V(4).Infof("Intermediate certificate cache hit (positive) for signer issuer %q", signer.Issuer)
		return obj.(*intermediateCertCacheEntry).pool, nil
	}
	if _, ok, _ := negative.GetByKey(keyStr); ok {
		klog.V(4).Infof("Intermediate certificate cache hit (negative) for signer issuer %q", signer.Issuer)
		return nil, fmt.Errorf("intermediate certificate fetch recently failed for signer issuer %q (cached)", signer.Issuer)
	}

	klog.V(2).Infof("Intermediate certificate cache miss for signer issuer %q", signer.Issuer)
	pool, fetchErr := fetch(signer)
	entry := &intermediateCertCacheEntry{key: keyStr, pool: pool}
	if fetchErr != nil {
		// List() walks every entry and lazily deletes expired ones; ListKeys() would not trigger
		// expiration. Doing this before each write bounds cache memory to ~(write_rate × TTL) without a
		// background goroutine, which matters most for the negative cache since an attacker rotating
		// issuer keys can drive writes to it at the fetch rate. Cost is O(N) per write, so in attack
		// conditions writes become slower as the cache grows, which also acts as a natural rate limit.
		// For legitimate traffic (a handful of entries), this is effectively free.
		_ = negative.List()
		_ = negative.Add(entry)
		return nil, fetchErr
	}
	// Evict expired entries before writing (same rationale as negative cache above).
	_ = positive.List()

View on GitHub (pinned to 4c8573c808)

Solutions

Wait for the negative cache TTL to expire and retry
Force a cache invalidation / restart the component to clear the negative entry
Fix the underlying fetch failure (network reachability, DNS, CA endpoint health) so the next fetch succeeds and populates the positive cache
Check whether the signer's AIA URL is correct and reachable from the cluster

Example fix

// before
// retrying immediately returns the cached negative error
pool, err := intermediateCertPoolForSigner(signer)
// after
if err != nil && strings.Contains(err.Error(), "(cached)") {
    time.Sleep(negativeCacheTTL) // or invalidate cache entry, then retry
    pool, err = intermediateCertPoolForSigner(signer)
}

Defensive patterns

Strategy: retry

Validate before calling

// Optionally peek at whether a negative entry exists before calling, if the caches are shared
if _, ok, _ := negativeCache.GetByKey(intermediateCacheKeyForSigner(signer)); ok {
    time.Sleep(negativeCacheTTL) // or skip/requeue the operation
}

Try / catch

pool, err := intermediateCertPoolForSigner(signer)
if err != nil && strings.Contains(err.Error(), "(cached)") {
    // back off until the negative TTL lapses, then retry once
    time.Sleep(negativeCacheTTL)
    pool, err = intermediateCertPoolForSigner(signer)
}
if err != nil { return nil, err }

Prevention

Fix the root fetch failure instead of retrying inside the negative-cache window
Provide an escape hatch to invalidate negative cache entries after remediation
Keep negative TTL short so transient outages self-heal

When it happens

Trigger: Calling intermediateCertPoolForSigner for a signer whose issuer previously failed an AIA fetch within the negative-cache TTL, and no positive cache entry exists (positive entries always win over stale negatives).

Common situations: Transient network outage to the CA's AIA URL; the CA endpoint returning non-200; DNS failures; repeated retries during the negative-cache window all fail fast with this cached error.

Understand the failure class

SSL/TLS and certificate errors — how TLS handshakes and certificate validation fail.

Related errors

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/abd07ace42e5637f. Report an issue: GitHub.