thanos-io/thanos · error

setup gRPC server

Error message

setup gRPC server

What it means

In runQuery's gRPC server setup block, tls.NewServerConfig builds the server TLS configuration from --grpc-server-tls-cert, --grpc-server-tls-key, and --grpc-server-tls-client-ca; any failure (unreadable files, bad PEM, invalid CA, unsupported min version/cipher/curve names) is wrapped as "setup gRPC server".

Solutions

1. Verify the cert, key, and client-CA file paths exist and are readable by the Thanos process.
2. Check the files are valid PEM (openssl x509 -in cert -noout).
3. Validate --grpc-server-tls-min-version, ciphers, and curves against supported values (e.g. TLS12).
4. If TLS is not needed, remove the TLS flags entirely.
5. Check mounted secret names/paths in your deployment manifest.

Example fix

// before
--grpc-server-tls-cert=/etc/certs/server.crt  // wrong mount path
// after
--grpc-server-tls-cert=/etc/thanos/tls/server.crt

for _, f := range []string{certPath, keyPath, caPath} {
    if f == "" { continue }
    fi, err := os.Stat(f)
    if err != nil || fi.IsDir() {
        return fmt.Errorf("TLS file missing: %s", f)
    }
    if b, err := os.ReadFile(f); err != nil || !bytes.Contains(b, []byte("-----BEGIN")) {
        return fmt.Errorf("TLS file not PEM: %s", f)
    }
}

if _, err := tls.NewServerConfig(logger, cert, key, ca, "", "", ""); err != nil {
    return fmt.Errorf("TLS setup failed: %w", err)
}

When it happens

Trigger: Calling `thanos query` with any of the TLS gRPC server flags set while the referenced cert/key/CA files are missing, unreadable, not valid PEM, or the --grpc-server-tls-min-version/ciphers/curves values are unrecognized.

Common situations: Kubernetes secret mounted at a different path than the flag, cert files with wrong permissions, expired/reformatted certificates, or typo'd cipher-suite names.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• building gRPC client
• setup gRPC server
• setup gRPC server
• setup gRPC server
• TSDB statistics not implemented

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/2ed6d3b4e6a858a4. Report an issue: GitHub.

Appendix: source

Thrown at cmd/thanos/query.go:624

		)
		srv.Handle("/", router)

		g.Add(func() error {
			statusProber.Healthy()

			return srv.ListenAndServe()
		}, func(err error) {
			statusProber.NotReady(err)
			defer statusProber.NotHealthy(err)

			srv.Shutdown(err)
		})
	}
	// Start query (proxy) gRPC StoreAPI.
	{
		tlsCfg, err := tls.NewServerConfig(log.With(logger, "protocol", "gRPC"), grpcServerConfig.tlsSrvCert, grpcServerConfig.tlsSrvKey, grpcServerConfig.tlsSrvClientCA, grpcServerConfig.tlsMinVersion, grpcServerConfig.tlsCiphers, grpcServerConfig.tlsCurves)
		if err != nil {
			return errors.Wrap(err, "setup gRPC server")
		}

		infoSrv := info.NewInfoServer(
			component.Query.String(),
			info.WithLabelSetFunc(func() []labelpb.ZLabelSet { return proxyStore.LabelSet() }),
			info.WithStoreInfoFunc(func() (*infopb.StoreInfo, error) {
				if httpProbe.IsReady() {
					mint, maxt := proxyStore.TimeRange()
					return &infopb.StoreInfo{
						MinTime:                      mint,
						MaxTime:                      maxt,
						SupportsSharding:             true,
						SupportsWithoutReplicaLabels: true,
						TsdbInfos:                    proxyStore.TSDBInfos(),
					}, nil
				}
				return nil, errors.New("Not ready")
			}),

View on GitHub (pinned to 35b8b99117)