thanos-io/thanos · error

setup gRPC server

Error message

setup gRPC server

What it means

Before starting the rule component's gRPC server, runRule builds a TLS server config via tls.NewServerConfig from --grpc-server-tls-cert/key/client-ca and cipher/curve flags. Any failure loading or validating these (missing files, bad PEM, mismatched key, unsupported cipher) is wrapped as "setup gRPC server" and stops startup before the listener is created.

Solutions

1. Verify --grpc-server-tls-cert and --grpc-server-tls-key point to existing, readable PEM files and that the pair matches (compare modulus/public key).
2. Validate the CA file and remove --grpc-server-tls-client-ca if mTLS is not intended.
3. Use only supported cipher names / TLS versions in the tls-ciphers and tls-min-version flags, or drop them for defaults.
4. Check that mounted Kubernetes secrets contain the expected keys (tls.crt/tls.key) and are correctly mounted.

Example fix

// before
thanos rule --grpc-server-tls-cert=/etc/ssl/wrong.crt --grpc-server-tls-key=/etc/ssl/server.key
// after
thanos rule --grpc-server-tls-cert=/etc/ssl/server.crt --grpc-server-tls-key=/etc/ssl/server.key

// before start: openssl x509 -in cert.pem -noout -checkend 0 && openssl x509 -in cert.pem -pubkey -noout | sha256sum && openssl pkey -in key.pem -pubout | sha256sum  # hashes must match

When it happens

Trigger: tls.NewServerConfig returns an error when: cert or key file paths don't exist or are unreadable; the cert/key pair doesn't match; PEM data is malformed; a configured TLS cipher suite or min version is invalid or unsupported; client CA file can't be parsed when mTLS is requested.

Common situations: Typo in --grpc-server-tls-cert path in a Kubernetes manifest; secret mounted with wrong key names; expired or rotated certificate mounted as the wrong file; specifying a TLS 1.0-era cipher with TLS 1.3 min version.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• setup gRPC server
• setup gRPC server
• create syncer
• building gRPC client
• endpoint

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/b80b6fdb5387adb0. Report an issue: GitHub.

Appendix: source

Thrown at cmd/thanos/rule.go:751

				}
			}
		}, func(error) {
			cancel()
		})
	}

	grpcProbe := prober.NewGRPC()
	httpProbe := prober.NewHTTP()
	statusProber := prober.Combine(
		httpProbe,
		grpcProbe,
		prober.NewInstrumentation(comp, logger, extprom.WrapRegistererWithPrefix("thanos_", reg)),
	)

	// Start gRPC server.
	tlsCfg, err := tls.NewServerConfig(log.With(logger, "protocol", "gRPC"), conf.grpc.tlsSrvCert, conf.grpc.tlsSrvKey, conf.grpc.tlsSrvClientCA, conf.grpc.tlsMinVersion, conf.grpc.tlsCiphers, conf.grpc.tlsCurves)
	if err != nil {
		return errors.Wrap(err, "setup gRPC server")
	}

	options := []grpcserver.Option{
		grpcserver.WithServer(thanosrules.RegisterRulesServer(ruleMgr)),
		grpcserver.WithListen(conf.grpc.bindAddress),
		grpcserver.WithGracePeriod(conf.grpc.gracePeriod),
		grpcserver.WithGracePeriod(conf.grpc.maxConnectionAge),
		grpcserver.WithTLSConfig(tlsCfg),
	}
	infoOptions := []info.ServerOptionFunc{info.WithRulesInfoFunc()}
	if tsdbDB != nil {
		tsdbStore := store.NewTSDBStore(logger, tsdbDB, component.Rule, conf.lset)
		infoOptions = append(
			infoOptions,
			info.WithLabelSetFunc(func() []labelpb.ZLabelSet {
				return tsdbStore.LabelSet()
			}),
			info.WithStoreInfoFunc(func() (*infopb.StoreInfo, error) {

View on GitHub (pinned to 35b8b99117)