{"record":{"id":"cb9c05e55cf43caa","repo":"thanos-io/thanos","slug":"remote-query-warning-s-s","errorCode":null,"errorMessage":"remote query warning (%s): %s","messagePattern":"remote query warning \\((.+?)\\): (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"pkg/query/remote_engine.go","lineNumber":380,"sourceCode":"\t\t\treturn r.responseForError(err)\n\t\t}\n\t\tvar (\n\t\t\tresult   = make(promql.Vector, 0)\n\t\t\twarnings annotations.Annotations\n\t\t\tbuilder  = labels.NewScratchBuilder(8)\n\t\t\tqryStats querypb.QueryStats\n\t\t)\n\t\tfor {\n\t\t\tmsg, err := qry.Recv()\n\t\t\tif err == io.EOF {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\tif err != nil {\n\t\t\t\treturn r.responseForError(err)\n\t\t\t}\n\n\t\t\tif warn := msg.GetWarnings(); warn != \"\" {\n\t\t\t\twarnings.Add(errors.Errorf(\"remote query warning (%s): %s\", r.remoteAddr, warn))\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif s := msg.GetStats(); s != nil {\n\t\t\t\tqryStats = *s\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif batch := msg.GetTimeseriesBatch(); batch != nil {\n\t\t\t\tfor _, ts := range batch.Series {\n\t\t\t\t\tbuilder.Reset()\n\t\t\t\t\tfor _, l := range ts.Labels {\n\t\t\t\t\t\tbuilder.Add(strings.Clone(l.Name), strings.Clone(l.Value))\n\t\t\t\t\t}\n\t\t\t\t\tif len(ts.Histograms) > 0 {\n\t\t\t\t\t\tresult = append(result, promql.Sample{Metric: builder.Labels(), H: prompb.FromProtoHistogram(ts.Histograms[0]), T: r.start.UnixMilli()})\n\t\t\t\t\t} else {\n\t\t\t\t\t\tresult = append(result, promql.Sample{Metric: builder.Labels(), F: ts.Samples[0].Value, T: r.start.UnixMilli()})\n\t\t\t\t\t}\n\t\t\t\t}","sourceCodeStart":362,"sourceCodeEnd":398,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/query/remote_engine.go#L362-L398","documentation":"While streaming a remote instant-query response, the remote endpoint sent a warnings field in a response message. The client records it as 'remote query warning (<remoteAddr>): <warn>' and continues consuming the stream; the query result carries these warnings as annotations, not an error.","triggerScenarios":"During remoteQuery.Exec for an instant query, a streamed response message has non-empty msg.GetWarnings() and no transport error.","commonSituations":"Remote querier executed the query with warnings: too many samples, histogram/NaN caveats, clock-skew hints, or partial data notices from downstream stores.","solutions":["Read the warning text after the address prefix; address the remote-side cause if the results are unreliable.","Treat results as potentially incomplete; tighten fanout/limits on the remote querier to eliminate the warning.","If warnings must fail the query, disable partial response handling on the remote engine.","Check remote store/querier logs for the same warning to find the originating store."],"exampleFix":"// warnings are annotations; inspect them after Exec\nres := qry.Exec(ctx)\nfor _, w := range res.Warnings {\n    log.Warnf(\"remote warning: %v\", w)\n}","handlingStrategy":"type-guard","validationCode":null,"typeGuard":"func hasRemoteWarnings(res *promql.Result) bool {\n    for _, w := range res.Warnings {\n        if strings.Contains(w.Error(), \"remote query warning\") {\n            return true\n        }\n    }\n    return false\n}","tryCatchPattern":"res := qry.Exec(ctx)\nif len(res.Warnings) > 0 {\n    log.Warnf(\"query completed with warnings: %v\", res.Warnings)\n}","preventionTips":["Log and alert on result warnings from remote queries.","Tune remote querier limits to reduce warning-generating conditions.","Pin Thanos versions on both ends to keep warning semantics consistent."],"tags":["remote-query","warnings","streaming","promql"],"backgroundTag":"upstream-api-error","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}