{"record":{"id":"eee122788e7d9d33","repo":"thanos-io/thanos","slug":"expected-d-in-config-payload-but-got-d-instead","errorCode":null,"errorMessage":"expected %d in config payload, but got %d instead","messagePattern":"expected (.+?) in config payload, but got (.+?) instead","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/discovery/memcache/resolver.go","lineNumber":95,"sourceCode":"\t\treturn nil, fmt.Errorf(\"failed to parse config size from metadata: %s, error: %s\", configMeta, err)\n\t}\n\n\tconfigVersion, err := reader.ReadString('\\n')\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to find config version: %s\", err)\n\t}\n\tclusterConfig.version, err = strconv.Atoi(strings.TrimSpace(configVersion))\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to parser config version: %s\", err)\n\t}\n\n\tnodes, err := reader.ReadString('\\n')\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to read nodes: %s\", err)\n\t}\n\n\tif len(configVersion)+len(nodes) != configSize {\n\t\treturn nil, fmt.Errorf(\"expected %d in config payload, but got %d instead\", configSize, len(configVersion)+len(nodes))\n\t}\n\n\tfor host := range strings.SplitSeq(strings.TrimSpace(nodes), \" \") {\n\t\tdnsIpPort := strings.Split(host, \"|\")\n\t\tif len(dnsIpPort) != 3 {\n\t\t\treturn nil, fmt.Errorf(\"node not in expected format: %s\", dnsIpPort)\n\t\t}\n\t\tport, err := strconv.Atoi(dnsIpPort[2])\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"failed to parse port: %s, err: %s\", dnsIpPort, err)\n\t\t}\n\t\tclusterConfig.nodes = append(clusterConfig.nodes, node{dns: dnsIpPort[0], ip: dnsIpPort[1], port: port})\n\t}\n\n\treturn clusterConfig, nil\n}\n","sourceCodeStart":77,"sourceCodeEnd":112,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/discovery/memcache/resolver.go#L77-L112","documentation":"The final validation in parseConfig: the version line plus nodes line must total exactly the byte count declared as configSize in the metadata header. When the declared length differs from what was actually read, the payload is incomplete or misframed and cannot be trusted, so this error is returned. It protects against accepting partially delivered or corrupted cluster membership data.","triggerScenarios":"The server writes fewer or more payload bytes than declared in 'CONFIG cluster 0 <len>' — truncation by a proxy, line-ending length accounting bugs (\\n vs \\r\\n), or a concurrent cluster reconfiguration changing the payload mid-write.","commonSituations":"ElastiCache failover or node add/remove racing with a fetch; custom memcached-compatible servers with off-by-N size computation; load balancer idle timeouts cutting the response.","solutions":["Fix the server-side size computation so configSize equals the exact byte length of the version+nodes payload.","Check for proxies/load balancers that could truncate or alter the response body.","Retry the fetch with backoff — transient inconsistencies occur during cluster reconfiguration.","Capture traffic (tcpdump) and compare declared vs actual payload bytes to locate the off-by-N."],"exampleFix":"// before\nfmt.Fprintf(conn, \"CONFIG cluster 0 %d\\r\\n%s\\r\\n\", len(body)+2, body) // overcounts\n// after\nbody := versionLine + nodesLine\nfmt.Fprintf(conn, \"CONFIG cluster 0 %d\\r\\n%s\", len(body), body)","handlingStrategy":"retry","validationCode":"func validatePayloadSize(headerLen int, versionLine, nodesLine string) error {\n    if got := len(versionLine) + len(nodesLine); got != headerLen {\n        return fmt.Errorf(\"expected %d payload bytes, got %d\", headerLen, got)\n    }\n    return nil\n}","typeGuard":null,"tryCatchPattern":"cfg, err := discovery.parseConfig(reader)\nif err != nil && strings.Contains(err.Error(), \"in config payload, but got\") {\n    return backoff.Retry(func() error { cfg, err = refetchConfig(addr); return err }, b)\n}","preventionTips":["Make the server the single source of truth for the declared length.","Check proxies/load balancers for response truncation or rewriting.","Retry with backoff; treat length mismatches as potentially transient during cluster changes.","Capture tcpdump traces to compare declared vs actual payload bytes."],"tags":["memcached","protocol","framing","validation"],"backgroundTag":"checksum-mismatch","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}