weaviate/weaviate · error

close %s: %w

Error message

close %s: %w

What it means

After successfully creating a recovery sentinel file, writeRecoveryTidiedSentinels closes it. If f.Close() fails (deferred flush/FS error, e.g. ENOSPC or EIO surfaced on close), the error is wrapped with the sentinel name and aborts FinalizeCompletedMigrations.

Source

Thrown at adapters/repos/db/inverted_reindex_finalize.go:393

// dataset; FinalizeCompletedMigrations needs the swapped/tidied
// sentinels in order to drive its existing ingest→canonical rename
// path. Writing them retroactively is safe because no buckets are
// loaded yet (we are pre-shard-init) and the underlying invariant
// (ingest dir holds the right data) has been verified by the
// `markMerged` semantics. We do NOT write swapped-per-prop sentinels
// because the existing finalize loop does not consume them.
func writeRecoveryTidiedSentinels(migDir string) error {
	for _, name := range []string{"swapped.mig", "tidied.mig"} {
		p := filepath.Join(migDir, name)
		if fileExists(p) {
			continue
		}
		f, err := os.OpenFile(p, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o644)
		if err != nil {
			return fmt.Errorf("create %s: %w", name, err)
		}
		if err := f.Close(); err != nil {
			return fmt.Errorf("close %s: %w", name, err)
		}
	}
	return nil
}

// removeStaleSidecarsForGen removes the `__<...>_<gen>` sidecar dirs
// (reindex/ingest/backup) belonging to an older, superseded generation
// of a finalized migration. Looks up the per-strategy suffix bases via
// `migrationSuffixes` (which now returns the suffix bases without the
// `_<N>` part) and removes any matching dir for the specific `_<gen>`.
//
// Props are read from the older gen's `properties.mig` (or recovered
// from the on-disk dirs themselves if properties.mig is missing — the
// latter is defensive against partial pre-migration state).
func removeStaleSidecarsForGen(lsmPath, namespace, dirName string, logger logrus.FieldLogger) {
	migDir := filepath.Join(lsmPath, ".migrations", dirName)
	suffixes := migrationSuffixes(dirName)
	if suffixes == nil {

View on GitHub (pinned to 75aa4b6d11)

Solutions

1. Free disk space and retry node startup/migration
2. Check disk/volume health (dmesg, smart data) for I/O errors
3. The sentinel file may exist but be suspect — verify it and recreate if empty/corrupt
4. Move data to reliable local storage rather than flaky network mounts

// pre-check storage health
if err := checkDiskWritable(dataDir); err != nil { return err }

if err := finalize(ctx); err != nil {
	if strings.Contains(err.Error(), "close ") {
		// EIO/ENOSPC on close: fix storage, verify/recreate sentinel, restart node
	}
}

When it happens

Trigger: Close-time flush fails due to disk full or I/O error; the file descriptor became invalid; underlying storage fault on the data volume.

Common situations: Disk full at exactly the sentinel-write moment; failing disk or network storage (EIO); container storage driver issues.

Related errors

• failed saving migration state
• failed creating migration skip flag
• failed to read runtime config
• create backup staging dir: %w
• copy inactive shard %s file %s to staging: %w

AI-assisted analysis of weaviate/weaviate@75aa4b6d11 (2026-09-04). Data as JSON: /api/errors/5029127c02e124aa. Report an issue: GitHub.