JuliusBrussee/caveman · error

objectstore: list page

Error message

objectstore: list page %q: %w

What it means

ListPage fetches maxKeys+1 keys via ListObjects (StartAfter, MaxKeys) to detect an extra page; this error wraps obj.Err from that listing stream. As with List, listing errors surface lazily during iteration, aborting the page. The prefix and cause are preserved with %w.

Solutions

1. Unwrap and check minio.ErrorResponse for AccessDenied (grant s3:ListBucket) or SlowDown (add backoff between pages).
2. Retry the page request on transient failures — pagination is stateless thanks to startAfter.
3. Verify startAfter is a valid key within the prefix ordering.
4. Reduce maxKeys if the backend caps MaxKeys and errors on larger values.
5. Check for infinite pagination loops (startAfter not advancing) that exhaust rate limits.

Example fix

// before
for {
    page, err := store.ListPage(ctx, prefix, cursor, 1000)
    if err != nil { return err }
    cursor = page.Next
}
// after
for {
    page, err := store.ListPage(ctx, prefix, cursor, 1000)
    if err != nil {
        return backoff.Retry(ctx, func() error {
            _, rerr := store.ListPage(ctx, prefix, cursor, 1000)
            return rerr
        })
    }
    if page.Next == "" || page.Next == cursor { break }
    cursor = page.Next
}

page, err := store.ListPage(ctx, prefix, startAfter, maxKeys)
if err != nil {
    var mr minio.ErrorResponse
    if errors.As(err, &mr) && (mr.Code == "SlowDown" || mr.Code == "RequestTimeout") {
        return retryWithBackoff(ctx, func() error {
            page, err = store.ListPage(ctx, prefix, startAfter, maxKeys)
            return err
        })
    }
    return err
}

When it happens

Trigger: Calling ListPage(ctx, prefix, startAfter, maxKeys) when the bounded listing fails: AccessDenied on s3:ListBucket, throttling (SlowDown), connection drop mid-page, or a backend that mishandles MaxKeys/StartAfter combinations.

Common situations: StartAfter values that don't match the prefix's key layout (some backends error or misbehave); tight pagination loops hitting rate limits; S3-compatible stores with MaxKeys caps lower than requested.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

• objectstore: list
• objectstore: list versions
• objectstore: delete
• objectstore: delete version
• objectstore: read

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/d5efc88c3133dc18. Report an issue: GitHub.

Appendix: source

Thrown at shared/platform/objectstore/objectstore.go:381

	}
	return keys, nil
}

func (m *minioStore) ListPage(ctx context.Context, prefix, startAfter string, maxKeys int) (ObjectPage, error) {
	if maxKeys <= 0 {
		return ObjectPage{}, fmt.Errorf("objectstore: page size must be positive")
	}
	// Request one extra key so the caller can tell whether another page exists,
	// while cancelling the producer as soon as the bounded result is known.
	pageCtx, cancel := context.WithCancel(ctx)
	defer cancel()
	keys := make([]string, 0, maxKeys)
	var extra string
	for obj := range m.client.ListObjects(pageCtx, m.bucket, minio.ListObjectsOptions{
		Prefix: prefix, Recursive: true, StartAfter: startAfter, MaxKeys: maxKeys + 1,
	}) {
		if obj.Err != nil {
			return ObjectPage{}, fmt.Errorf("objectstore: list page %q: %w", prefix, obj.Err)
		}
		if len(keys) < maxKeys {
			keys = append(keys, obj.Key)
		} else {
			extra = obj.Key
			break
		}
	}
	if extra == "" {
		return ObjectPage{Keys: keys}, nil
	}
	return ObjectPage{Keys: keys, NextToken: keys[len(keys)-1], Truncated: true}, nil
}

View on GitHub (pinned to 3ee70a1026)