{"record":{"id":"4a0a331f84e23e0d","repo":"thanos-io/thanos","slug":"server-s","errorCode":null,"errorMessage":"server=%s","messagePattern":"server=(.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"internal/cortex/chunk/cache/memcached_client.go","lineNumber":225,"sourceCode":"\t// Skip hitting memcached at all if the item is bigger than the max allowed size.\n\tif c.maxItemSize > 0 && len(item.Value) > c.maxItemSize {\n\t\tc.skipped.Inc()\n\t\treturn nil\n\t}\n\n\terr := c.Client.Set(item)\n\tif err == nil {\n\t\treturn nil\n\t}\n\n\t// Inject the server address in order to have more information about which memcached\n\t// backend server failed. This is a best effort.\n\taddr, addrErr := c.serverList.PickServer(item.Key)\n\tif addrErr != nil {\n\t\treturn err\n\t}\n\n\treturn errors.Wrapf(err, \"server=%s\", addr)\n}\n\nfunc (c *memcachedClient) updateLoop(updateInterval time.Duration) {\n\tdefer c.wait.Done()\n\tticker := time.NewTicker(updateInterval)\n\tfor {\n\t\tselect {\n\t\tcase <-ticker.C:\n\t\t\terr := c.updateMemcacheServers()\n\t\t\tif err != nil {\n\t\t\t\tlevel.Warn(c.logger).Log(\"msg\", \"error updating memcache servers\", \"err\", err)\n\t\t\t}\n\t\tcase <-c.quit:\n\t\t\tticker.Stop()\n\t\t\treturn\n\t\t}\n\t}\n}","sourceCodeStart":207,"sourceCodeEnd":243,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/internal/cortex/chunk/cache/memcached_client.go#L207-L243","documentation":"memcachedClient.Set wraps an error returned by the memcached set operation with the address of the backend server that was picked for the key. The wrap preserves the original error (timeout, server down, etc.) and annotates it with the failing server's address for debugging.","triggerScenarios":"Calling Set (via the cache Store path) after PickServer succeeded, but the actual memcached SET command failed (connection error, timeout, server error response).","commonSituations":"Memcached restarted or evicting connections mid-operation; network partition to one memcached node; item larger than the server's item size limit; timeouts under load.","solutions":["Inspect the wrapped original error to determine the failure class (timeout vs connection vs server error).","Check the health of the memcached server named in the message (server=<addr>).","Set a sane memcached timeout and enable the client's server-update loop so the server list stays fresh.","Retry the store; Set failures are best-effort in this cache and non-fatal to query serving."],"exampleFix":"// before: hard to tell which node failed\nreturn err\n// after: error now annotated\nreturn errors.Wrapf(err, \"server=%s\", addr)","handlingStrategy":"retry","validationCode":"if err := pingMemcached(addr, 500*time.Millisecond); err != nil { log.Warnf(\"memcached %s unhealthy before Set: %v\", addr, err) }","typeGuard":null,"tryCatchPattern":"err := c.Set(ctx, key, val, ttl)\nif err != nil {\n    var se *memcachedServerError\n    if errors.As(err, &se) { /* server rejected item */ } else { /* transient: retry once or degrade */ }\n    log.Warnf(\"cache set failed (best-effort): %v\", err)\n}","preventionTips":["Keep server lists fresh (updateLoop) and set reasonable timeouts.","Keep cached items below the memcached item size limit.","Treat Set failures as non-fatal; the cache is best-effort."],"tags":["memcached","network","cache"],"backgroundTag":"http-request-failed","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}