{"record":{"id":"829a94bc92582a60","repo":"weaviate/weaviate","slug":"failed-to-read-certificate-from-s-w","errorCode":null,"errorMessage":"failed to read certificate from %s: %w","messagePattern":"failed to read certificate from (.+?): %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"usecases/auth/authentication/oidc/middleware.go","lineNumber":409,"sourceCode":"// S3 URI, or inline PEM string) and returns a certificate pool containing it.\n// Note: HTTP URL fetches use the default http.Client, so the certificate URL\n// must be reachable without custom TLS settings. Certificate and SkipTLSVerify\n// are mutually exclusive, so this function is only called when SkipTLSVerify\n// is false.\nfunc (c *Client) loadCertPool() (*x509.CertPool, error) {\n\tvar certificate, certificateSource string\n\tif strings.HasPrefix(c.Config.Certificate.Get(), \"http\") {\n\t\tresp, err := http.Get(c.Config.Certificate.Get())\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"failed to get certificate from %s: %w\", c.Config.Certificate.Get(), err)\n\t\t}\n\t\tdefer resp.Body.Close()\n\t\tif resp.StatusCode != http.StatusOK {\n\t\t\treturn nil, fmt.Errorf(\"failed to download certificate from %s: http status: %v\", c.Config.Certificate.Get(), resp.StatusCode)\n\t\t}\n\t\tcertBytes, err := io.ReadAll(resp.Body)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"failed to read certificate from %s: %w\", c.Config.Certificate.Get(), err)\n\t\t}\n\t\tcertificate = string(certBytes)\n\t\tcertificateSource = c.Config.Certificate.Get()\n\t} else if strings.HasPrefix(c.Config.Certificate.Get(), \"s3://\") {\n\t\tparts := strings.TrimPrefix(c.Config.Certificate.Get(), \"s3://\")\n\t\tsegments := strings.SplitN(parts, \"/\", 2)\n\t\tif len(segments) != 2 {\n\t\t\treturn nil, fmt.Errorf(\"invalid S3 URI, must contain bucket and key: %s\", c.Config.Certificate.Get())\n\t\t}\n\t\tregion := os.Getenv(\"AWS_REGION\")\n\t\tif region == \"\" {\n\t\t\tregion = os.Getenv(\"AWS_DEFAULT_REGION\")\n\t\t}\n\t\tcreds := credentials.NewIAM(\"\")\n\t\t// check if we are able to get the credentials using AWS IAM\n\t\tif _, err := creds.GetWithContext(nil); err != nil {\n\t\t\t// if IAM doesn't work, check environment settings for creds, set anonymous access if none found\n\t\t\tcreds = credentials.NewEnvAWS()","sourceCodeStart":391,"sourceCodeEnd":427,"githubUrl":"https://github.com/weaviate/weaviate/blob/75aa4b6d11f8818305aafd4440b4e32794f7ca04/usecases/auth/authentication/oidc/middleware.go#L391-L427","documentation":"Once the certificate bytes are obtained (HTTP or S3 source), loadCertPool reads the body; an io.ReadAll / io.Copy failure wraps as 'failed to read certificate from <url>'. This means the response started successfully but the body could not be fully consumed.","triggerScenarios":"loadCertPool's io.ReadAll(resp.Body) (HTTP path) or io.Copy into a buffer (S3 path) returns an error mid-stream: connection reset while streaming, truncated response, S3 object read failure.","commonSituations":"Flaky network or proxy dropping long bodies; S3 object removed mid-download or credentials lacking read permission surfacing as a read error via minio; server closing connections early under load.","solutions":["Retry the fetch; if intermittent, it's a network/proxy stability issue.","If the source is S3, verify the object exists and the IAM credentials have s3:GetObject.","Check proxy/LB timeouts and increase body-size/timeout limits.","Fall back to a locally mounted certificate file to remove the network dependency."],"exampleFix":"// before (S3 read failing mid-stream)\nAUTHENTICATION_OIDC_CERTIFICATE=s3://my-bucket/ca.pem\n// after (local mount)\nAUTHENTICATION_OIDC_CERTIFICATE=/etc/weaviate/certs/ca.pem","handlingStrategy":"retry","validationCode":"// Prefetch the full body to validate readability before configuring\nresp, err := http.Get(certURL)\nif err != nil { return err }\nbody, err := io.ReadAll(resp.Body)\nresp.Body.Close()\nif err != nil { return fmt.Errorf(\"cert body not readable: %w\", err) }\nif len(body) == 0 { return fmt.Errorf(\"empty certificate\") }","typeGuard":null,"tryCatchPattern":"const maxRetries = 3\nfor i := 0; i < maxRetries; i++ {\n\tif _, err := client.Init(ctx); err == nil || !strings.Contains(err.Error(), \"failed to read certificate\") {\n\t\tbreak\n\t}\n\ttime.Sleep(time.Second << i)\n}","preventionTips":["Prefer local file mounts to remove streaming bodies over the network","Increase proxy/LB read timeouts for large objects","Ensure S3 credentials permit the full object read"],"tags":["go","io","network","oidc","s3"],"backgroundTag":"request-body-read-failed","analyzedSha":"75aa4b6d11f8818305aafd4440b4e32794f7ca04","analyzedAt":"2026-09-04T14:58:20.392Z","contentChangedAt":"2026-09-04T14:58:20.392Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}