ipfs/kubo · warning

iterating datastore keys: %w

Error message

iterating datastore keys: %w

What it means

Raised by deleteStaleDHTValueRecords while consuming the results channel from a keys-only datastore Query: an individual result carries result.Error, meaning the datastore iteration hit an error mid-scan (e.g. a key/value entry is unreadable). The purge stops at that point, returning the count of keys deleted so far; the outer purgeStaleDHTValueRecords logs a warning and retries on next start.

Source

Thrown at core/node/storage.go:219

// deleteStaleDHTValueRecords scans the whole datastore (keys only) and
// deletes old-format DHT value records in batches. It returns the number of
// deleted keys.
func deleteStaleDHTValueRecords(ctx context.Context, ds datastore.Batching) (int, error) {
	results, err := ds.Query(ctx, query.Query{Prefix: "/", KeysOnly: true})
	if err != nil {
		return 0, fmt.Errorf("querying datastore for stale DHT value records: %w", err)
	}
	defer results.Close()

	var batch datastore.Batch
	var count, pending int
	for result := range results.Next() {
		if ctx.Err() != nil {
			return count, ctx.Err()
		}
		if result.Error != nil {
			return count, fmt.Errorf("iterating datastore keys: %w", result.Error)
		}
		if !isStaleDHTValueRecordKey(result.Key) {
			continue
		}
		if batch == nil {
			batch, err = ds.Batch(ctx)
			if err != nil {
				return count, fmt.Errorf("creating delete batch: %w", err)
			}
		}
		if err := batch.Delete(ctx, datastore.NewKey(result.Key)); err != nil {
			return count, fmt.Errorf("batch deleting stale key %s: %w", result.Key, err)
		}
		count++
		pending++
		if pending >= purgeBatchSize {
			if err := batch.Commit(ctx); err != nil {
				return count, fmt.Errorf("committing delete batch: %w", err)

View on GitHub (pinned to 329838acdf)

Solutions

  1. Identify the failing key/cause from the wrapped result.Error in logs
  2. Run the datastore's repair/verification tooling (e.g. badger flush/check)
  3. Back up the repo and consider datastore rebuild if corruption persists
  4. Restart the daemon — purge resumes/retries on next start
Defensive patterns

Strategy: retry

Type guard

if result.Error != nil {
    return fmt.Errorf("iterating datastore keys: %w", result.Error)
}

Try / catch

count, err := deleteStaleDHTValueRecords(ctx, ds)
if err != nil {
    log.Warnw("purge incomplete", "deletedSoFar", count, "error", err)
    // retried automatically on next daemon start
}

Prevention

When it happens

Trigger: Mid-iteration datastore failure during the stale DHT value record purge — corrupted record in the datastore, iterator I/O error, or underlying database error surfaced per-result.

Common situations: Badger corruption /checksum mismatch on some entries after unclean shutdown, failing disk sectors, or datastore backend bugs.

Related errors


AI-assisted analysis of ipfs/kubo@329838acdf (2026-09-03). Data as JSON: /api/errors/aefa48de2507e0b8. Report an issue: GitHub.