{"record":{"id":"e25ffde956f7bc0a","repo":"thanos-io/thanos","slug":"not-uploading-as-no-external-labels-are-configured","errorCode":null,"errorMessage":"not uploading as no external labels are configured yet - is Prometheus healthy/reachable?","messagePattern":"not uploading as no external labels are configured yet - is Prometheus healthy/reachable\\?","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"cmd/thanos/sidecar.go","lineNumber":453,"sourceCode":"\t\t\t\trunutil.CloseWithLogOnErr(logger, bkt, \"bucket client\")\n\t\t\t}\n\t\t}()\n\n\t\tif err := promclient.IsWALDirAccessible(conf.tsdb.path); err != nil {\n\t\t\tlevel.Error(logger).Log(\"err\", err)\n\t\t}\n\n\t\tctx, cancel := context.WithCancel(context.Background())\n\t\tg.Add(func() error {\n\t\t\tdefer runutil.CloseWithLogOnErr(logger, bkt, \"bucket client\")\n\n\t\t\tpromReadyTimeout := conf.prometheus.readyTimeout\n\t\t\textLabelsCtx, cancel := context.WithTimeout(ctx, promReadyTimeout)\n\t\t\tdefer cancel()\n\n\t\t\tif err := runutil.Retry(2*time.Second, extLabelsCtx.Done(), func() error {\n\t\t\t\tif m.Labels().Len() == 0 {\n\t\t\t\t\treturn errors.New(\"not uploading as no external labels are configured yet - is Prometheus healthy/reachable?\")\n\t\t\t\t}\n\t\t\t\treturn nil\n\t\t\t}); err != nil {\n\t\t\t\treturn errors.Wrapf(err, \"aborting as no external labels found after waiting %s\", promReadyTimeout)\n\t\t\t}\n\n\t\t\tdataDir, err := os.OpenRoot(conf.tsdb.path)\n\t\t\tif err != nil {\n\t\t\t\treturn errors.Wrap(err, \"opening tsdb directory\")\n\t\t\t}\n\t\t\tdefer runutil.CloseWithLogOnErr(logger, dataDir, \"tsdb directory\")\n\n\t\t\ts := shipper.New(\n\t\t\t\tbkt,\n\t\t\t\tdataDir,\n\t\t\t\tshipper.WithLogger(logger),\n\t\t\t\tshipper.WithRegisterer(reg),\n\t\t\t\tshipper.WithSource(metadata.SidecarSource),","sourceCodeStart":435,"sourceCodeEnd":471,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/cmd/thanos/sidecar.go#L435-L471","documentation":"This error is returned (and retried) inside the sidecar's main run loop when the Prometheus external-labels provider reports zero labels. Thanos requires external labels (e.g. prometheus=<cluster>) to stamp blocks before uploading to object storage, so it keeps retrying for promReadyTimeout and fails if Prometheus never provides them.","triggerScenarios":"m.Labels().Len() == 0 at the start of sidecar operation: Prometheus has no external_labels configured in its config, or the sidecar cannot yet scrape/fetch them from the configured Prometheus URL within the retry window.","commonSituations":"Fresh Prometheus install without external_labels section; sidecar started before Prometheus finished loading config; misconfigured --prometheus.url so labels endpoint queries the wrong instance; Prometheus not yet healthy after startup.","solutions":["Add an external_labels section (e.g. external_labels: {prometheus: <name>, replica: <n>}) to prometheus.yml and reload Prometheus.","Verify the sidecar's --prometheus.url points at the correct Prometheus instance.","Increase --prometheus.ready-timeout if Prometheus legitimately takes long to become ready.","Check Prometheus /-/healthy and that the labels endpoint responds before the timeout elapses."],"exampleFix":"# before (prometheus.yml)\nglobal:\n  scrape_interval: 15s\n# after\nglobal:\n  scrape_interval: 15s\n  external_labels:\n    prometheus: eu1-prod\n    replica: 0","handlingStrategy":"validation","validationCode":"curl -s <promURL>/api/v1/status/config | grep -A3 external_labels","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always set external_labels (prometheus + replica) in every prometheus.yml used with Thanos.","Add external-label presence to config-management linting.","Start sidecars only after Prometheus reports /-/healthy."],"tags":["prometheus","thanos-sidecar","configuration","external-labels"],"backgroundTag":"missing-required-config","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}