weaviate/weaviate · error

shard %q: remove computed usage file for unloaded shard: %w

Error message

shard %q: remove computed usage file for unloaded shard: %w

What it means

Wraps an error from shardusage.RemoveComputedUsageDataForUnloadedShard during shard initialization. Before a shard is opened, Weaviate deletes any stale computed-usage file for that (not yet loaded) shard; if removal fails, NewShard aborts so the shard never comes up in an inconsistent state. The wrapped error names the filesystem-level cause.

Source

Thrown at adapters/repos/db/shard_init.go:53

	"github.com/weaviate/weaviate/entities/storagestate"
	"github.com/weaviate/weaviate/usecases/monitoring"
)

func NewShard(ctx context.Context, promMetrics *monitoring.PrometheusMetrics,
	shardName string, index *Index, class *models.Class, jobQueueCh chan job,
	scheduler *queue.Scheduler, indexCheckpoints *indexcheckpoint.Checkpoints,
	reindexer ShardReindexerV3, lazyLoadSegments bool, bitmapBufPool roaringset.BitmapBufPool,
	registration monitoring.ShardRegistration,
) (_ *Shard, err error) {
	start := time.Now()
	index.logger.WithFields(logrus.Fields{
		"action": "init_shard",
		"shard":  shardName,
		"index":  index.ID(),
	}).Debugf("initializing shard %q", shardName)

	if err := shardusage.RemoveComputedUsageDataForUnloadedShard(index.path(), shardName); err != nil {
		return nil, fmt.Errorf("shard %q: remove computed usage file for unloaded shard: %w", shardName, err)
	}

	if err := newPropertyDeleteIndexHelper().ensureBucketsAreRemovedForNonExistentPropertyIndexes(index.path(), shardName, class); err != nil {
		return nil, fmt.Errorf("shard %q: remove nonexistent property index buckets: %w", shardName, err)
	}

	if err := newVectorDropIndexHelper().ensureFilesAreRemovedForDroppedVectorIndexes(index.path(), shardName, class); err != nil {
		return nil, fmt.Errorf("shard %q: remove dropped vector index files: %w", shardName, err)
	}

	metrics, err := NewMetrics(index.logger, promMetrics, string(index.Config.ClassName), shardName)
	if err != nil {
		return nil, fmt.Errorf("init shard %q metrics: %w", shardName, err)
	}
	if index.Config.LazySegmentsDisabled {
		lazyLoadSegments = false // disabled globally
	}

View on GitHub (pinned to 75aa4b6d11)

Solutions

1. Inspect the wrapped error to identify the filesystem cause (permission denied, read-only fs, etc.).
2. Fix permissions/ownership on the Weaviate data directory (chown to the user running the weaviate process).
3. Ensure the volume is mounted read-write and the filesystem is not full or out of inodes.
4. Check for immutable attributes on the usage file (chattr -i) or locks from other processes.
5. If a specific file is unremovable, manually delete the computed usage file for that shard while Weaviate is stopped, then restart.

Example fix

// before: data dir owned by root while weaviate runs as user 999
// ls -l /var/lib/weaviate -> root:root
// after
docker exec -u root weaviate chown -R 999:999 /var/lib/weaviate

// pre-flight check before starting/upgrading weaviate
dir := "/var/lib/weaviate"
info, err := os.Stat(dir)
if err != nil { panic(err) }
if info.Mode().Perm()&0200 == 0 {
    panic("weaviate data dir is not writable")
}
probe := filepath.Join(dir, ".write-test")
if err := os.WriteFile(probe, []byte("x"), 0o644); err != nil {
    panic(fmt.Sprintf("data dir not writable: %v", err))
}
os.Remove(probe)

// shard init errors are fatal at startup; guard the readiness check
if err := client.Readiness().Do(ctx); err != nil {
    if strings.Contains(err.Error(), "remove computed usage file for unloaded shard") {
        // stop, fix filesystem permissions/ownership, restart
        os.Exit(1)
    }
}

When it happens

Trigger: Calling NewShard (shard creation/open, including during initShard and replica snapshot creation) where deleting the computed usage file under the shard's directory fails — e.g. filesystem permission error, read-only mount, or the path being an unremovable directory.

Common situations: Container/VM volume mounted read-only; wrong ownership or permissions on /var/lib/weaviate after migration or restore from backup; disk full (rare for unlink) or immutable files; running as a non-root user against root-owned data after a docker permission change.

Related errors

• sweep changelog dir for shard %q: %w
• create backup staging dir: %w
• stat shard dir: %w
• create staging subdir for inactive shard %s file %s: %w
• stat objects bucket dir for shard %s: %w

AI-assisted analysis of weaviate/weaviate@75aa4b6d11 (2026-09-04). Data as JSON: /api/errors/4442efc4935c7b66. Report an issue: GitHub.