{"record":{"id":"6b6430f8bc02f837","repo":"thanos-io/thanos","slug":"check-exists","errorCode":null,"errorMessage":"check exists","messagePattern":"check exists","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/shipper/shipper.go","lineNumber":413,"sourceCode":"\t\t}\n\n\t\tif m.Stats.NumSamples == 0 {\n\t\t\t// Ignore empty blocks.\n\t\t\tlevel.Debug(s.logger).Log(\"msg\", \"ignoring empty block\", \"block\", m.ULID)\n\t\t\tcontinue\n\t\t}\n\n\t\t// We only ship of the first compacted block level as normal flow.\n\t\tif m.Compaction.Level > 1 {\n\t\t\tif !s.uploadCompacted {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t}\n\n\t\t// Check against bucket if the meta file for this block exists.\n\t\tok, err := s.bucket.Exists(ctx, path.Join(m.ULID.String(), block.MetaFilename))\n\t\tif err != nil {\n\t\t\treturn uploaded, errors.Wrap(err, \"check exists\")\n\t\t}\n\t\tif ok {\n\t\t\tmeta.Uploaded = append(meta.Uploaded, m.ULID)\n\t\t\tcontinue\n\t\t}\n\n\t\t// Skip overlap check if out of order uploads is enabled.\n\t\tif m.Compaction.Level > 1 && !s.allowOutOfOrderUploads {\n\t\t\tif err := checker.IsOverlapping(ctx, m.BlockMeta); err != nil {\n\t\t\t\treturn uploaded, errors.Errorf(\"Found overlap or error during sync, cannot upload compacted block, details: %v\", err)\n\t\t\t}\n\t\t}\n\n\t\tif err := s.upload(ctx, m); err != nil {\n\t\t\tif !s.allowOutOfOrderUploads {\n\t\t\t\treturn uploaded, errors.Wrapf(err, \"upload %v\", m.ULID)\n\t\t\t}\n","sourceCodeStart":395,"sourceCodeEnd":431,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/shipper/shipper.go#L395-L431","documentation":"Wraps an error from s.bucket.Exists(ctx, <ulid>/meta.json) during Sync: the bucket existence check for a block's meta file failed. Sync checks the bucket before uploading each block to skip already-uploaded ones; a failed Exists call (network, auth, object store error) aborts the remaining sync with the partial 'uploaded' list.","triggerScenarios":"Any Sync invocation where the object storage backend returns an error for the Exists request — e.g. bucket does not exist, expired credentials, DNS/network failure, throttling, or misconfigured objstore endpoint.","commonSituations":"Wrong bucket name or region in objstore config; IAM/AccessKey revoked or clock skew breaking S3 signatures; GCS/S3 throttling (rate limits); network partition between sidecar and object storage.","solutions":["Inspect the wrapped cause — it names the backend (s3/gcs/azure) and the underlying error; fix that first.","Verify objstore.config credentials and bucket name/region with `thanos tools bucket verify` or a direct listing.","Check network connectivity/DNS and any proxy/firewall between the sidecar and the object store endpoint.","Rotate expired credentials and confirm IAM permissions (s3:HeadObject / storage.objects.get).","Retry — Sync is idempotent and safe to re-run; consider increasing provider timeouts if throttled."],"exampleFix":"// before\n// check exists: s3: HeadObject https://s3.eu-west-1.amazonaws.com/thanos-bucket/01ARZ.../meta.json: AccessDenied\n// after (fix IAM policy)\n// {\"Effect\":\"Allow\",\"Action\":[\"s3:ListBucket\",\"s3:GetObject\",\"s3:PutObject\",\"s3:DeleteObject\"],\"Resource\":[\"arn:aws:s3:::thanos-bucket\",\"arn:aws:s3:::thanos-bucket/*\"]}","handlingStrategy":"retry","validationCode":"cfg, err := objstore.ClientFromConfig(bucketCfg)\nif err != nil { return err }\nctx, cancel := context.WithTimeout(ctx, 30*time.Second)\ndefer cancel()\nif _, err := cfg.Iter(ctx, \"\", func(string) error { return nil }); err != nil {\n    return fmt.Errorf(\"bucket preflight failed: %w\", err)\n}","typeGuard":null,"tryCatchPattern":"if _, err := shipper.Sync(ctx); err != nil {\n    if strings.Contains(err.Error(), \"check exists\") {\n        // backend auth/network issue; back off and retry with jitter\n        return retry.WithBackoff(ctx, syncFn)\n    }\n}","preventionTips":["Preflight-verify bucket credentials and name before deploying the sidecar","Rotate credentials before expiry; keep NTP-synced clocks for S3 signatures","Set sane timeouts and retries in the objstore config","Alert on object-store rate limits/throttling metrics"],"tags":["object-storage","s3","network","sidecar"],"backgroundTag":"http-error-response","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}