weaviate/weaviate · error

open: not found at original or delete-marker path: %w

Error message

open: not found at original or delete-marker path: %w

What it means

This error is returned by openWithDeleteMarkerFallback in usecases/backup/zip.go when, during a backup write, neither the original relative path nor the delete-marker-derived path (entBackup.DeleteMarkerAdd(relPath)) could be opened under the backup source directory. It wraps the underlying os.Open error, so the cause is almost always a missing file in the source tree. The library throws it because backup streaming expects every file listed in the iteration to exist either at its original location or at its delete-marker location.

Source

Thrown at usecases/backup/zip.go:506

			}
			return info, nil
		}
		return nil, fmt.Errorf("stat: %w", err)
	}
	return info, nil
}

// openWithDeleteMarkerFallback opens the file at relPath under the source
// directory. If the file does not exist, it retries with the delete-marker
// prefix in case the collection was renamed during an ongoing backup.
func (z *zip) openWithDeleteMarkerFallback(relPath string) (*os.File, error) {
	absPath := filepath.Join(z.sourcePath, relPath)
	f, err := os.Open(absPath)
	if err != nil {
		if os.IsNotExist(err) {
			f, err = os.Open(filepath.Join(z.sourcePath, entBackup.DeleteMarkerAdd(relPath)))
			if err != nil {
				return nil, fmt.Errorf("open: not found at original or delete-marker path: %w", err)
			}
			return f, nil
		}
		return nil, fmt.Errorf("open: %w", err)
	}
	return f, nil
}

type zstdWrapper struct {
	z *zstd.Decoder
}

func (z zstdWrapper) Read(p []byte) (n int, err error) {
	return z.z.Read(p)
}

func (z zstdWrapper) Close() error {
	z.z.Close()

View on GitHub (pinned to 75aa4b6d11)

Solutions

  1. Verify the file exists at z.sourcePath/<relPath>; if not, check for a delete-marker-named variant and restore whichever is missing from a prior backup or the live shard.
  2. Check that the backup sourcePath configuration points to the correct, complete backup destination directory.
  3. Re-run the backup/restore operation against a source directory that is not being concurrently modified; pause deletion/cleanup jobs during backup.
  4. Inspect the wrapped error in the message (%w) for the concrete path and syscall failure (e.g. no such file or directory vs permission denied).

Example fix

// before (opaque failure)
f, err := openWithDeleteMarkerFallback(relPath)
if err != nil { return err }

// after (skip-and-log files that vanished mid-backup)
f, err := openWithDeleteMarkerFallback(relPath)
if err != nil {
    var pe *fs.PathError
    if errors.As(err, &pe) && errors.Is(pe, fs.ErrNotExist) {
        logger.Warnf("skipping vanished backup file: %v", err)
        return nil // continue with next file
    }
    return err
}
Defensive patterns

Strategy: try-catch

Validate before calling

p := filepath.Join(sourcePath, relPath)
if _, err := os.Stat(p); err != nil {
    if _, err2 := os.Stat(filepath.Join(sourcePath, entBackup.DeleteMarkerAdd(relPath))); err2 != nil {
        return fmt.Errorf("backup source incomplete, missing %s: %w", relPath, err2)
    }
}

Try / catch

if err != nil {
    var pe *fs.PathError
    if errors.As(err, &pe) && errors.Is(pe.Err, fs.ErrNotExist) {
        // reconcile: re-list backup contents or abort cleanly
        logger.Warnf("backup file vanished: %v", err)
        return errSkipFile
    }
    return err
}

Prevention

When it happens

Trigger: WriteRegular calls openWithDeleteMarkerFallback with a relPath enumerated from the backup contents, but the file was removed from the source path between listing and opening, or the delete-marker companion file is also absent (e.g. partially deleted source directory, incomplete backup destination).

Common situations: Source data deleted while a backup is running (concurrent deletion), restoring from a source directory that was pruned by retention/cleanup, misconfigured sourcePath pointing at the wrong or truncated directory, NFS/shared-filesystem sync issues where the delete-marker file was not yet flushed.

Related errors

AI-assisted analysis of weaviate/weaviate@75aa4b6d11 (2026-09-04). Data as JSON: /api/errors/9b5a06903b295d38. Report an issue: GitHub.