JuliusBrussee/caveman · error · ErrTooLarge

objectstore: object exceeds read limit

Error message

objectstore: object exceeds read limit

What it means

objectstore.ErrTooLarge is returned by Get when the bytes read from the bucket exceed the configured in-memory read ceiling (maxBytes). Artifact and capture payloads are bounded on write, but this read-side ceiling additionally prevents a corrupted or attacker-written bucket object from exhausting the service process's memory.

Solutions

1. Verify the object's expected size and raise the configured read limit if the payload legitimately needs to be larger.
2. Stream the object instead of reading it fully into memory if the store API allows it.
3. Audit the bucket for out-of-band writes; enforce write-side bounds so only bounded payloads ever land in the bucket.
4. Investigate possible tampering if the size is unexpected for a key that should be bounded.

Example fix

// before
body, err := objectstore.GetBounded(ctx, store, key, 1<<20) // 1 MiB limit too small
// after
body, err := objectstore.GetBounded(ctx, store, key, 16<<20) // 16 MiB limit
if errors.Is(err, objectstore.ErrTooLarge) {
    return nil, fmt.Errorf("artifact %s exceeds read limit", key)
}

if size > maxBytes { return fmt.Errorf("object %s too large for read limit", key) }

body, err := store.Get(ctx, key)
if errors.Is(err, objectstore.ErrTooLarge) {
    return nil, fmt.Errorf("object %s exceeds read ceiling", key)
}

When it happens

Trigger: Calling Get/GetObject on an object whose byte length exceeds the maxBytes limit passed to the read helper (objectstore.go:301); a bucket object was written outside normal write bounds, tampered with, or corrupted to be oversized; the caller supplied too small a maxBytes for a legitimately large object.

Common situations: Loading an artifact that legitimately grew larger than the configured limit; reading an object uploaded manually or by a third party into the shared bucket; investigating a hostile bucket where someone wrote a huge object under a known key.

Understand the failure class

Background: payload too large / request exceeds maximum size: why libraries cap bytes and how to fix oversize payloads — this error's family across 50 libraries.

Related errors

• objectstore: object not found
• objectstore: production read/write/delete probe failed
• probe bytes did not round-trip
• gzip close
• gzip

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/ebdd2bd68603fae6. Report an issue: GitHub.

Appendix: source

Thrown at shared/platform/objectstore/objectstore.go:35

	"fmt"
	"io"
	"net/url"
	"strings"
	"time"

	"github.com/JuliusBrussee/caveman/shared/platform/env"
	"github.com/minio/minio-go/v7"
	"github.com/minio/minio-go/v7/pkg/credentials"
)

// ErrNotFound is returned by Get/Delete when the key does not exist.
var ErrNotFound = errors.New("objectstore: object not found")

// ErrTooLarge is returned when a remote object exceeds the configured in-memory
// read ceiling. Artifact/capture payloads are bounded on write; enforcing a
// read ceiling too prevents a corrupted or attacker-written bucket object from
// exhausting a service process.
var ErrTooLarge = errors.New("objectstore: object exceeds read limit")

// ErrPagingUnsupported is returned when a caller that must bound listing
// memory is given a backend without a cursor-capable implementation.
var ErrPagingUnsupported = errors.New("objectstore: paged listing unsupported")

// Store is the blob backend contract.
type Store interface {
	// Put writes body at key, overwriting any existing object.
	Put(ctx context.Context, key string, body []byte, contentType string) error
	// Get returns the bytes at key, or ErrNotFound.
	Get(ctx context.Context, key string) ([]byte, error)
	// Delete removes key (no error if already absent).
	Delete(ctx context.Context, key string) error
	// Exists reports whether key is present.
	Exists(ctx context.Context, key string) (bool, error)
	// List returns all keys under prefix (used by retention + ZDR assertions).
	List(ctx context.Context, prefix string) ([]string, error)
}

View on GitHub (pinned to 3ee70a1026)