{"record":{"id":"7289c5d4c9b0da71","repo":"thanos-io/thanos","slug":"out-of-order-labels","errorCode":null,"errorMessage":"out of order labels","messagePattern":"out of order labels","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/store/labelpb/label.go","lineNumber":26,"sourceCode":"\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"sort\"\n\t\"strings\"\n\t\"sync\"\n\t\"unique\"\n\t\"unsafe\"\n\n\t\"github.com/VictoriaMetrics/easyproto\"\n\t\"github.com/cespare/xxhash/v2\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/prometheus/prometheus/model/labels\"\n)\n\nvar (\n\tErrOutOfOrderLabels = errors.New(\"out of order labels\")\n\tErrEmptyLabels      = errors.New(\"label set contains a label with empty name or value\")\n\tErrDuplicateLabels  = errors.New(\"label set contains duplicate label names\")\n\n\tsep = []byte{'\\xff'}\n)\n\nfunc noAllocString(buf []byte) string {\n\treturn *(*string)(unsafe.Pointer(&buf))\n}\n\nfunc noAllocBytes(buf string) []byte {\n\treturn *(*[]byte)(unsafe.Pointer(&buf))\n}\n\n// ZLabelsFromPromLabels converts Prometheus labels to slice of labelpb.ZLabel in type unsafe manner.\n// It reuses the same memory. Caller should abort using passed labels.Labels.\nfunc ZLabelsFromPromLabels(lset labels.Labels) []ZLabel {\n\treturn *(*[]ZLabel)(unsafe.Pointer(&lset))","sourceCodeStart":8,"sourceCodeEnd":44,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/store/labelpb/label.go#L8-L44","documentation":"ErrOutOfOrderLabels is a sentinel error declared in pkg/store/labelpb signaling that a label set's labels are not sorted in lexicographic order by name, which the columnar/capnp proto encoding requires. Writers like the receive component's capnproto writer return it when consecutive label names compare in the wrong order during validation or writing.","triggerScenarios":"Writing a TimeSeries whose labels are not sorted by name (e.g. [zone=a, env=b]) via the capnproto writer (pkg/receive/capnproto_writer.go validateLabels/addLabelsError), or decoding series data that violates the ordering invariant.","commonSituations":"Custom relabeling or external code appending labels in arbitrary order before remote-write ingestion; hand-built labels.Labels not passed through labels.New(); forwarding writes between Thanos Receive clients with differently-ordered label sets.","solutions":["Sort the label set by label name before writing (labels.New builds a sorted set; use labels.Labels.Range after sorting).","Ensure relabeling/external-label injection keeps labels lexicographically ordered.","Use compare/merge helpers (labels.Labels.Compare) in custom writers to enforce ordering."],"exampleFix":"// before\nlbls := labels.FromStrings(\"zone\", \"a\", \"app\", \"b\") // unsorted\n// after\nlbls := labels.FromStrings(\"app\", \"b\", \"zone\", \"a\") // sorted by name","handlingStrategy":"validation","validationCode":"func labelsSorted(lbls labels.Labels) bool {\n\treturn lbls.Validate(func(l labels.Label) error { return nil }) == nil &&\n\t\tfunc() bool {\n\t\t\tprev := \"\"\n\t\t\tok := true\n\t\t\tlbls.Range(func(l labels.Label) error {\n\t\t\t\tif l.Name <= prev { ok = false }\n\t\t\t\tprev = l.Name\n\t\t\t\treturn nil\n\t\t\t})\n\t\t\treturn ok\n\t\t}()\n}","typeGuard":"func isOutOfOrderLabels(err error) bool { return errors.Is(err, labelpb.ErrOutOfOrderLabels) }","tryCatchPattern":"if err := writer.Write(ctx, series); err != nil {\n\tif errors.Is(err, labelpb.ErrOutOfOrderLabels) {\n\t\t// sort labels and retry once\n\t\tsorted := labels.NewBuilder(series.Labels).Labels()\n\t\tseries.Labels = sorted\n\t\treturn writer.Write(ctx, series)\n\t}\n\treturn err\n}","preventionTips":["Always build label sets with labels.New/labels.FromStrings with names in sorted order.","Run relabeling output through labels.Builder which maintains ordering.","Add unit tests asserting lexicographic label order for hand-built series."],"tags":["labels","validation","remote-write"],"backgroundTag":"invalid-argument-format","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}