{"record":{"id":"edbbb7d11ed4c6c2","repo":"thanos-io/thanos","slug":"found-that-prometheus-and-thanos-use-different-pat","errorCode":null,"errorMessage":"found that Prometheus and Thanos use different paths for tracking block uploads. Prometheus uses --storage.tsdb.delay-compact-file.path=%s while Thanos will write to %s, they must both use the same path.","messagePattern":"found that Prometheus and Thanos use different paths for tracking block uploads\\. Prometheus uses --storage\\.tsdb\\.delay-compact-file\\.path=(.+?) while Thanos will write to (.+?), they must both use the same path\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cmd/thanos/sidecar.go","lineNumber":520,"sourceCode":"\tif err := runutil.Retry(2*time.Second, ctx.Done(), func() error {\n\t\tif flags, flagErr = client.ConfiguredFlags(ctx, m.promURL); flagErr != nil && flagErr != promclient.ErrFlagEndpointNotFound {\n\t\t\tlevel.Warn(logger).Log(\"msg\", \"failed to get Prometheus flags. Is Prometheus running? Retrying\", \"err\", flagErr)\n\t\t\treturn errors.Wrapf(flagErr, \"fetch Prometheus flags\")\n\t\t}\n\t\treturn nil\n\t}); err != nil {\n\t\treturn errors.Wrapf(err, \"fetch Prometheus flags\")\n\t}\n\n\tif flagErr != nil {\n\t\tlevel.Warn(logger).Log(\"msg\", \"failed to check Prometheus flags, due to potentially older Prometheus. No extra validation is done.\", \"err\", flagErr)\n\t\treturn nil\n\t}\n\n\tif flags.TSDBDelayCompact != \"\" {\n\t\tthanosMetaPath := filepath.Join(conf.tsdb.path, conf.shipper.metaFileName)\n\t\tif filepath.Clean(flags.TSDBDelayCompact) != filepath.Clean(thanosMetaPath) {\n\t\t\treturn errors.Errorf(\n\t\t\t\t\"found that Prometheus and Thanos use different paths for tracking block uploads. \"+\n\t\t\t\t\t\"Prometheus uses --storage.tsdb.delay-compact-file.path=%s while Thanos will write to %s, they must both use the same path.\",\n\t\t\t\tflags.TSDBDelayCompact, thanosMetaPath,\n\t\t\t)\n\t\t}\n\t}\n\n\t// Check if compaction is disabled.\n\tif flags.TSDBMinTime != flags.TSDBMaxTime && flags.TSDBDelayCompact == \"\" {\n\t\tif !conf.shipper.ignoreBlockSize {\n\t\t\treturn errors.Errorf(\"found that TSDB Max time is %s and Min time is %s. \"+\n\t\t\t\t\"Compaction needs to be disabled (storage.tsdb.min-block-duration = storage.tsdb.max-block-duration)\", flags.TSDBMaxTime, flags.TSDBMinTime)\n\t\t}\n\t\tlevel.Warn(logger).Log(\"msg\", \"flag to ignore Prometheus min/max block duration flags differing is being used. If the upload of a 2h block fails and a Prometheus compaction happens that block may be missing from your Thanos bucket storage.\")\n\t}\n\t// Check if block time is 2h.\n\tif flags.TSDBMinTime != model.Duration(2*time.Hour) {\n\t\tlevel.Warn(logger).Log(\"msg\", \"found that TSDB block time is not 2h. Only 2h block time is recommended.\", \"block-time\", flags.TSDBMinTime)","sourceCodeStart":502,"sourceCodeEnd":538,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/cmd/thanos/sidecar.go#L502-L538","documentation":"validatePrometheus rejects the configuration when Prometheus's --storage.tsdb.delay-compact-file.path flag points to a different file than the path where the Thanos shipper writes its meta.json. Both processes must track delayed-compaction block uploads at the same path, otherwise upload tracking diverges.","triggerScenarios":"flags.TSDBDelayCompact is non-empty and filepath.Clean(Prometheus flag value) != filepath.Clean(conf.tsdb.path + shipper.metaFileName); i.e. the two configured paths differ after normalization.","commonSituations":"Operator set --storage.tsdb.delay-compact-file.path in Prometheus but did not align --tsdb.path/--shipper.meta-file-name on the Thanos side; relative vs absolute path confusion (though Clean is applied); copy-pasting the flag from another host.","solutions":["Make Prometheus's --storage.tsdb.delay-compact-file.path exactly equal to <thanos --tsdb.path>/<shipper meta file name> (default meta.json in the TSDB dir).","Or remove the delay-compact-file flag from Prometheus if the delayed-compaction feature is not intended.","Verify both resolved paths with realpath and compare."],"exampleFix":"# before\nprometheus --storage.tsdb.delay-compact-file.path=/prometheus/upload-tracking.json\nthanos sidecar --tsdb.path=/prometheus\n# after\nprometheus --storage.tsdb.delay-compact-file.path=/prometheus/meta.json\nthanos sidecar --tsdb.path=/prometheus","handlingStrategy":"validation","validationCode":"if promFlag != filepath.Clean(tsdbPath + \"/meta.json\") {\n    return errors.New(\"delay-compact-file.path must match thanos shipper meta path\")\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Generate the Prometheus delay-compact-file flag from the same variable that sets --tsdb.path.","Compare realpath of both paths in deployment CI.","Document the pairing of the two flags in your runbook."],"tags":["configuration","validation","thanos-sidecar","prometheus-flags"],"backgroundTag":"invalid-config-value","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}