{"record":{"id":"431aab8f57cab0bf","repo":"thanos-io/thanos","slug":"get-v-from-origin-bucket","errorCode":null,"errorMessage":"get %v from origin bucket","messagePattern":"get (.+?) from origin bucket","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/replicate/scheme.go","lineNumber":298,"sourceCode":"func (rs *replicationScheme) ensureObjectReplicated(ctx context.Context, objectName string) error {\n\tlevel.Debug(rs.logger).Log(\"msg\", \"ensuring object is replicated\", \"object\", objectName)\n\n\texists, err := rs.toBkt.Exists(ctx, objectName)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"check if %v exists in target bucket\", objectName)\n\t}\n\n\t// skip if already exists.\n\tif exists {\n\t\tlevel.Debug(rs.logger).Log(\"msg\", \"skipping object as already replicated\", \"object\", objectName)\n\t\treturn nil\n\t}\n\n\tlevel.Debug(rs.logger).Log(\"msg\", \"object not present in target bucket, replicating\", \"object\", objectName)\n\n\tr, err := rs.fromBkt.Get(ctx, objectName)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"get %v from origin bucket\", objectName)\n\t}\n\n\tdefer r.Close()\n\n\tif err = rs.toBkt.Upload(ctx, objectName, r); err != nil {\n\t\treturn errors.Wrapf(err, \"upload %v to target bucket\", objectName)\n\t}\n\n\tlevel.Info(rs.logger).Log(\"msg\", \"object replicated\", \"object\", objectName)\n\trs.metrics.objectsReplicated.Inc()\n\n\treturn nil\n}\n","sourceCodeStart":280,"sourceCodeEnd":312,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/replicate/scheme.go#L280-L312","documentation":"ensureObjectReplicated failed to fetch an object from the origin bucket via rs.fromBkt.Get. If the source read fails, replication to the target bucket cannot proceed and the error is wrapped with the object name.","triggerScenarios":"Block replication calls ensureObjectReplicated for an object name; the object bucket Get fails with NotFound (object deleted/GC'd from source), permissions error, or backend connectivity error.","commonSituations":"Compactor removed the block between listing and fetching; wrong bucket/endpoint credentials; object storage outage or throttling; object filtered by relabel still referenced.","solutions":["Check the source bucket still contains the object (mc ls / gsutil ls)","Verify fromBkt credentials and permissions (read access)","Retry — the replication loop re-runs and skips if the source block is intentionally gone (ignore deletion races)","Check object storage logs/endpoint reachability and rate limits"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// pre-check object presence in origin before replication\nif _, err := fromBkt.Exists(ctx, objectName); err != nil {\n    return fmt.Errorf(\"object %s not available in origin: %w\", objectName, err)\n}","typeGuard":null,"tryCatchPattern":"_, err := rs.fromBkt.Get(ctx, objectName)\nif err != nil {\n    if errors.Is(err, context.Canceled) { return err }\n    if isNotExist(err) {\n        // source block already deleted: skip, don't retry\n        return nil\n    }\n    return errors.Wrapf(err, \"get %v from origin bucket\", objectName)\n}","preventionTips":["Ensure replication doesn't race the compactor's deletion of source blocks","Validate bucket credentials with a cheap HEAD/List at startup","Distinguish NotFound from transient backend errors before retrying"],"tags":["object-storage","replication","thanos"],"backgroundTag":"resource-not-found","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}