{"record":{"id":"4956a7b724305cd3","repo":"JuliusBrussee/caveman","slug":"s-load-system-certificate-pool-w","errorCode":null,"errorMessage":"%s: load system certificate pool: %w","messagePattern":"(.+?): load system certificate pool: %w","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"shared/platform/chhttp/chhttp.go","lineNumber":167,"sourceCode":"// Appending (rather than replacing) keeps a public managed endpoint verifiable\n// while a private CA is trusted for the internal one.\n//\n// The bundle is parsed block by block instead of via CertPool.AppendCertsFromPEM,\n// which reports success as soon as ONE certificate parses and silently drops the\n// rest. A secret mount that is truncated mid-bundle, or corrupt after the first\n// entry, would then be half-trusted: the endpoints whose issuer survived keep\n// verifying and the ones whose issuer was dropped fail later, at the first\n// telemetry flush, looking like a network fault. Any unusable certificate block —\n// or a trailing PEM header with no complete block behind it — fails the whole\n// bundle CLOSED at boot instead.\nfunc rootsWithCAFile(path string) (*x509.CertPool, error) {\n\tbundle, err := os.ReadFile(path)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"%s: %w\", caFileEnv, err)\n\t}\n\troots, err := x509.SystemCertPool()\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"%s: load system certificate pool: %w\", caFileEnv, err)\n\t}\n\tadded := 0\n\trest := bundle\n\tfor {\n\t\tvar block *pem.Block\n\t\tblock, rest = pem.Decode(rest)\n\t\tif block == nil {\n\t\t\tbreak\n\t\t}\n\t\tif block.Type != \"CERTIFICATE\" {\n\t\t\tcontinue\n\t\t}\n\t\tcert, err := x509.ParseCertificate(block.Bytes)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"%s (%s): certificate %d is unparseable, so the bundle is incomplete and must not be half-trusted: %w\", caFileEnv, path, added+1, err)\n\t\t}\n\t\troots.AddCert(cert)\n\t\tadded++","sourceCodeStart":149,"sourceCodeEnd":185,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/27d5a3981a347890211bb1bf2439e5c821a63bc9/shared/platform/chhttp/chhttp.go#L149-L185","documentation":"Thrown by rootsWithCAFile in shared/platform/chhttp/chhttp.go:167 after the custom CA bundle file was read successfully but x509.SystemCertPool() failed. The function builds a trust pool by starting from the OS trust store and appending the bundle's certificates, so the system pool is a hard prerequisite. The error is prefixed with the CA-file environment variable name so the operator knows which knob produced it.","triggerScenarios":"Setting the CA-file env var (caFileEnv, e.g. CLICKHOUSE_CA_FILE) to a readable file while the process cannot load the operating system trust store: SSL_CERT_FILE or SSL_CERT_DIR points to a missing/unreadable file, or the container image ships no /etc/ssl/certs at all (scratch/distroless without ca-certificates).","commonSituations":"Minimal Docker images (FROM scratch or distroless) that never installed the ca-certificates package; a stray SSL_CERT_FILE env var left pointing at a file deleted during image slimming; hardened hosts where the trust store is not readable by the service user.","solutions":["Install the OS certificate package in the image (apk add ca-certificates / apt-get install ca-certificates) or copy /etc/ssl/certs from a builder stage.","Check SSL_CERT_FILE and SSL_CERT_DIR in the process environment and make sure they point to existing, readable files/directories.","Verify the service user has read permission on the system trust store.","As a last resort on scratch images, set SSL_CERT_FILE to the custom bundle itself so SystemCertPool reads a file that exists."],"exampleFix":"# before (Dockerfile)\nFROM scratch\nCOPY myapp /myapp\n\n# after\nFROM golang:1.23 AS build\nRUN go build -o /myapp ./...\nFROM alpine:3.20\nRUN apk add --no-cache ca-certificates\nCOPY --from=build /myapp /myapp","handlingStrategy":"validation","validationCode":"// before setting the CA-file env var, prove the system pool loads\nif _, err := x509.SystemCertPool(); err != nil {\n    log.Fatalf(\"system trust store unusable: %v (check SSL_CERT_FILE/SSL_CERT_DIR and image certs)\", err)\n}","typeGuard":null,"tryCatchPattern":"roots, err := chhttpTrustPool(path)\nif err != nil {\n    // fail closed: do not fall back to a default transport\n    return fmt.Errorf(\"boot: TLS trust config: %w\", err)\n}","preventionTips":["Install ca-certificates in every container image that runs the service.","Add a boot-time smoke test that loads the system pool in the deployed image, not just on dev machines.","Keep SSL_CERT_FILE/SSL_CERT_DIR pointing at existing files in production manifests."],"tags":["go","tls","certificates","config","containers"],"backgroundTag":null,"analyzedSha":"27d5a3981a347890211bb1bf2439e5c821a63bc9","analyzedAt":"2026-08-15T09:26:11.751Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}