What causes "failed to verify storage permissions: %w" in weaviate/weaviate?

Inspect the wrapped cause (%w chain) for the underlying storage error and fix credentials/bucket/permissions

failed to verify storage permissions: %w

Error message

failed to verify storage permissions: %w

What it means

When config.Usage.VerifyPermissions is enabled, InitializeCommon calls storage.VerifyPermissions(ctx) at startup to confirm the configured storage backend is accessible and writable. Any underlying failure is wrapped with this prefix so the operator sees a permission/verification failure.

Source

Thrown at usecases/modulecomponents/usage/base_module.go:156

	if b.config.Usage.ShardConcurrency != nil {
		if shardConcurrency := b.config.Usage.ShardConcurrency.Get(); shardConcurrency > 0 {
			b.shardConcurrency = shardConcurrency
		}
	}
	// push the parsed value in case the usage service was wired before Init
	if b.usageService != nil {
		b.usageService.SetShardConcurrency(b.shardConcurrency)
	}

	// Verify storage permissions (opt-in)
	var shouldVerifyPermissions bool
	if b.config.Usage.VerifyPermissions != nil {
		shouldVerifyPermissions = b.config.Usage.VerifyPermissions.Get()
	}

	if shouldVerifyPermissions {
		if err := b.storage.VerifyPermissions(ctx); err != nil {
			return fmt.Errorf("failed to verify storage permissions: %w", err)
		}
		b.logger.Info("storage permissions verified successfully")
	} else {
		b.logger.Info("storage permission verification skipped (disabled by configuration)")
	}

	// try to adjust the initial interval, to avoid push gaps after Weaviate's restarts
	if err := b.adjustInitialInterval(config); err != nil {
		b.logger.Errorf("cannot adjust initial interval, falling back to: %v: %v", b.interval, err)
	}

	b.logger.Infof("%s module initialized successfully", b.moduleName)
	return nil
}

func (b *BaseModule) collectAndUploadPeriodically(ctx context.Context) {
	// Validate intervals before creating tickers
	if b.interval <= 0 {

View on GitHub (pinned to 75aa4b6d11)

Solutions

Inspect the wrapped cause (%w chain) for the underlying storage error and fix credentials/bucket/permissions
Verify IAM policy grants the required read/write actions on the storage backend
Confirm bucket/container names and region in the storage config
If verification is intentionally not wanted, disable it via the VerifyPermissions config option

Example fix

// before
aws s3 ls s3://my-bucket  # AccessDenied — IAM lacks s3:ListBucket
// after
# attach policy granting s3:ListBucket and s3:GetObject/PutObject on my-bucket, then restart

Defensive patterns

Strategy: try-catch

Validate before calling

// pre-flight: verify storage credentials before starting
if _, err := store.Stat(ctx, probePath); err != nil {
  return fmt.Errorf("storage probe failed before init: %w", err)
}

Try / catch

if err := module.Init(ctx, cfg, logger, metrics); err != nil {
  var inner error
  if errors.As(err, &inner) || errors.Unwrap(err) != nil {
    logger.Fatalf("usage init failed: %v (cause: %v)", err, errors.Unwrap(err))
  }
}

Prevention

Verify storage credentials and IAM policies in CI before deploying
Test bucket/container access with the same role the server runs as
Read the wrapped cause chain — this error always wraps the real storage failure
Consider a pre-flight permissions probe at deploy time rather than at boot

When it happens

Trigger: Usage module with USE_USAGE_VERIFY_PERMISSIONS=true where the storage backend (e.g. object store) rejects the check due to bad credentials, missing bucket, or insufficient IAM permissions.

Common situations: Misconfigured cloud credentials (expired keys, wrong role/policy); bucket name typos; network/firewall blocking storage endpoint at boot.

Related errors

AI-assisted analysis of weaviate/weaviate@75aa4b6d11 (2026-09-04). Data as JSON: /api/errors/09afa2b61b707bb2. Report an issue: GitHub.