benbjohnson/litestream · error

validate level %d for %s: %w

Error message

validate level %d for %s: %w

What it means

Store.Validate failed while db.Replica.ValidateLevel(ctx, lvl.Level) fetched the LTX file listing for a level from replica storage. ValidateLevel performs integrity checks across a level's files; the error here means the listing itself could not be retrieved (as opposed to validation findings, which are returned in result.Errors). The wrapped error includes the level and database path.

Source

Thrown at store.go:881

// Validate checks LTX file consistency across all databases and levels.
// SnapshotLevel (9) is excluded since snapshots are not contiguous.
func (s *Store) Validate(ctx context.Context) (*ValidationResult, error) {
	result := &ValidationResult{Valid: true}

	s.mu.Lock()
	dbs := s.dbs
	levels := s.levels
	s.mu.Unlock()

	for _, db := range dbs {
		if db.Replica == nil {
			continue
		}

		for _, lvl := range levels {
			errs, err := db.Replica.ValidateLevel(ctx, lvl.Level)
			if err != nil {
				return nil, fmt.Errorf("validate level %d for %s: %w", lvl.Level, db.Path(), err)
			}
			if len(errs) > 0 {
				result.Valid = false
				result.Errors = append(result.Errors, errs...)
			}
		}
	}

	return result, nil
}

// monitorValidation periodically runs validation checks on all databases.
func (s *Store) monitorValidation(ctx context.Context) {
	s.Logger.Info("starting validation monitor", "interval", s.ValidationInterval)

	ticker := time.NewTicker(s.ValidationInterval)
	defer ticker.Stop()

View on GitHub (pinned to 4ed7a308f6)

Solutions

1. Read the wrapped error from ValidateLevel ("fetch ltx files: %w") for the storage-specific cause and fix credentials/bucket config.
2. Verify the replica storage location still exists and contains the expected level structure.
3. Re-run litestream validate (or wait for the next monitorValidation cycle) after restoring storage access.
4. Check context timeouts if validation runs against slow or remote storage.

Example fix

// before
errs, err := db.Replica.ValidateLevel(ctx, lvl.Level)
if err != nil {
    return nil, fmt.Errorf("validate level %d for %s: %w", lvl.Level, db.Path(), err)
}
// after
errs, err := db.Replica.ValidateLevel(ctx, lvl.Level)
if err != nil {
    if ctx.Err() != nil {
        return nil, fmt.Errorf("validation canceled: %w", ctx.Err())
    }
    return nil, fmt.Errorf("validate level %d for %s: %w", lvl.Level, db.Path(), err)
}

ctx, cancel := context.WithTimeout(ctx, 10*time.Minute)
defer cancel()
itr, err := client.LTXFiles(ctx, level, 0, false)
if err != nil {
    return fmt.Errorf("level %d not listable before validate: %w", level, err)
}
itr.Close()

results, err := store.Validate(ctx)
if err != nil {
    if errors.Is(err, context.DeadlineExceeded) || isStorageTransient(err) {
        log.Printf("validation deferred; will retry next cycle")
        return nil
    }
    return err
}

When it happens

Trigger: Validate(ctx) / monitorValidation iterating levels for each DB where Client.LTXFiles(ctx, level, ...) fails: auth failure, network error, missing bucket/prefix, or context timeout during enumeration.

Common situations: Storage credentials rotated or revoked before a scheduled validation run, bucket deleted or renamed, network partition, or provider throttling during large listings.

Understand the failure class

Background: 'Something went wrong' / 'Request failed (500)' / 'HTTP error! status: 404' — what failed HTTP requests actually mean and how to find the real cause — this error's family across 28 libraries.

Related errors

• fetch dst level info: %w
• fetch src level info: %w
• enforce snapshot retention: %w
• enforce L%d retention: %w
• snapshot interval must be greater than 0

AI-assisted analysis of benbjohnson/litestream@4ed7a308f6 (2026-09-06). Data as JSON: /api/errors/0273f4b14d0d70fa. Report an issue: GitHub.