What causes "provider instance is closed" in grpc/grpc-go?

Ensure no goroutines are calling KeyMaterial after Close; coordinate shutdown so in-flight handshakes complete first.

provider instance is closed

Error message

provider instance is closed

What it means

errProviderClosed is a sentinel error returned by certprovider.Distributor.KeyMaterial and by the closedProvider wrapper when the provider or distributor has been closed (Stop/Close called) and can no longer serve key material. It indicates the certificate provider's lifecycle has ended and subsequent calls for TLS key material will fail.

Solutions

Ensure no goroutines are calling KeyMaterial after Close; coordinate shutdown so in-flight handshakes complete first.
If using the certprovider store, rely on its reference counting rather than manually closing providers.
Handle errProviderClosed gracefully in credential wrappers (e.g., by falling back or returning a clear error to the caller).

Example fix

// before
km, err := provider.KeyMaterial(ctx)
if err != nil { log.Fatal(err) } // crashes on shutdown
// after
km, err := provider.KeyMaterial(ctx)
if errors.Is(err, certprovider.ErrProviderClosed) { // not exported; check message or stop first
    return status.Error(codes.Unavailable, "credential provider shutting down")
}

Defensive patterns

Strategy: try-catch

Validate before calling

// Track provider lifecycle: do not call KeyMaterial after Close.
// Use a done channel or context to coordinate shutdown.

Try / catch

km, err := provider.KeyMaterial(ctx)
if err != nil {
    if err.Error() == "provider instance is closed" {
        return status.Error(codes.Unavailable, "credential provider closed")
    }
    return err
}

Prevention

Use the certprovider store's reference counting rather than closing providers manually.
Coordinate shutdown so in-flight handshakes finish before Close.
Avoid long-lived goroutines that outlive the owning channel.

When it happens

Trigger: Calling KeyMaterial on a Provider (or its underlying Distributor) after Close() has been called. Also returned by closedProvider, which the store wraps providers in after their reference count drops to zero. This can happen if a channel is closed and garbage-collected while a goroutine still tries to read key material, or if the provider is explicitly closed too early.

Common situations: Race conditions during shutdown where a connection handshake is in progress when the provider is closed. Reference-counting bugs in the certprovider store where a provider is released while still referenced. Long-running goroutines that outlive the channel that owned the provider.

Related errors

AI-assisted analysis of grpc/grpc-go@0c51461d27 (2026-08-11). Data as JSON: /api/errors/ef77f3306c83fbd8. Report an issue: GitHub.

Appendix: source

Thrown at credentials/tls/certprovider/provider.go:44

import (
	"context"
	"crypto/tls"
	"crypto/x509"
	"errors"

	"github.com/spiffe/go-spiffe/v2/bundle/spiffebundle"
	"google.golang.org/grpc/internal"
)

func init() {
	internal.GetCertificateProviderBuilder = getBuilder
}

var (
	// errProviderClosed is returned by Distributor.KeyMaterial when it is
	// closed.
	errProviderClosed = errors.New("provider instance is closed")

	// m is a map from name to Provider builder.
	m = make(map[string]Builder)
)

// Register registers the Provider builder, whose name as returned by its Name()
// method will be used as the name registered with this builder. Registered
// Builders are used by the Store to create Providers.
func Register(b Builder) {
	m[b.Name()] = b
}

// getBuilder returns the Provider builder registered with the given name.
// If no builder is registered with the provided name, nil will be returned.
func getBuilder(name string) Builder {
	if b, ok := m[name]; ok {
		return b
	}

View on GitHub (pinned to 0c51461d27)