{"record":{"id":"ebdd2bd68603fae6","repo":"JuliusBrussee/caveman","slug":"objectstore-object-exceeds-read-limit","errorCode":null,"errorMessage":"objectstore: object exceeds read limit","messagePattern":"objectstore: object exceeds read limit","errorType":"exception","errorClass":"ErrTooLarge","httpStatus":null,"severity":"error","filePath":"shared/platform/objectstore/objectstore.go","lineNumber":35,"sourceCode":"\t\"fmt\"\n\t\"io\"\n\t\"net/url\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/JuliusBrussee/caveman/shared/platform/env\"\n\t\"github.com/minio/minio-go/v7\"\n\t\"github.com/minio/minio-go/v7/pkg/credentials\"\n)\n\n// ErrNotFound is returned by Get/Delete when the key does not exist.\nvar ErrNotFound = errors.New(\"objectstore: object not found\")\n\n// ErrTooLarge is returned when a remote object exceeds the configured in-memory\n// read ceiling. Artifact/capture payloads are bounded on write; enforcing a\n// read ceiling too prevents a corrupted or attacker-written bucket object from\n// exhausting a service process.\nvar ErrTooLarge = errors.New(\"objectstore: object exceeds read limit\")\n\n// ErrPagingUnsupported is returned when a caller that must bound listing\n// memory is given a backend without a cursor-capable implementation.\nvar ErrPagingUnsupported = errors.New(\"objectstore: paged listing unsupported\")\n\n// Store is the blob backend contract.\ntype Store interface {\n\t// Put writes body at key, overwriting any existing object.\n\tPut(ctx context.Context, key string, body []byte, contentType string) error\n\t// Get returns the bytes at key, or ErrNotFound.\n\tGet(ctx context.Context, key string) ([]byte, error)\n\t// Delete removes key (no error if already absent).\n\tDelete(ctx context.Context, key string) error\n\t// Exists reports whether key is present.\n\tExists(ctx context.Context, key string) (bool, error)\n\t// List returns all keys under prefix (used by retention + ZDR assertions).\n\tList(ctx context.Context, prefix string) ([]string, error)\n}","sourceCodeStart":17,"sourceCodeEnd":53,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/3ee70a102609e550bd2e68004bf5990a9341c851/shared/platform/objectstore/objectstore.go#L17-L53","documentation":"objectstore.ErrTooLarge is returned by Get when the bytes read from the bucket exceed the configured in-memory read ceiling (maxBytes). Artifact and capture payloads are bounded on write, but this read-side ceiling additionally prevents a corrupted or attacker-written bucket object from exhausting the service process's memory.","triggerScenarios":"Calling Get/GetObject on an object whose byte length exceeds the maxBytes limit passed to the read helper (objectstore.go:301); a bucket object was written outside normal write bounds, tampered with, or corrupted to be oversized; the caller supplied too small a maxBytes for a legitimately large object.","commonSituations":"Loading an artifact that legitimately grew larger than the configured limit; reading an object uploaded manually or by a third party into the shared bucket; investigating a hostile bucket where someone wrote a huge object under a known key.","solutions":["Verify the object's expected size and raise the configured read limit if the payload legitimately needs to be larger.","Stream the object instead of reading it fully into memory if the store API allows it.","Audit the bucket for out-of-band writes; enforce write-side bounds so only bounded payloads ever land in the bucket.","Investigate possible tampering if the size is unexpected for a key that should be bounded."],"exampleFix":"// before\nbody, err := objectstore.GetBounded(ctx, store, key, 1<<20) // 1 MiB limit too small\n// after\nbody, err := objectstore.GetBounded(ctx, store, key, 16<<20) // 16 MiB limit\nif errors.Is(err, objectstore.ErrTooLarge) {\n    return nil, fmt.Errorf(\"artifact %s exceeds read limit\", key)\n}","handlingStrategy":"validation","validationCode":"if size > maxBytes { return fmt.Errorf(\"object %s too large for read limit\", key) }","typeGuard":null,"tryCatchPattern":"body, err := store.Get(ctx, key)\nif errors.Is(err, objectstore.ErrTooLarge) {\n    return nil, fmt.Errorf(\"object %s exceeds read ceiling\", key)\n}","preventionTips":["Size your read ceiling above your write-side payload bounds","Stream large payloads instead of buffering when possible","Monitor bucket objects for out-of-band or oversized writes","Treat ErrTooLarge on a bounded key as a potential tampering signal"],"tags":["objectstore","s3","memory-limit","payload-size"],"backgroundTag":"payload-too-large","analyzedSha":"3ee70a102609e550bd2e68004bf5990a9341c851","analyzedAt":"2026-09-20T15:53:39.229Z","contentChangedAt":"2026-09-20T15:53:39.229Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}