{"record":{"id":"09afa2b61b707bb2","repo":"weaviate/weaviate","slug":"failed-to-verify-storage-permissions-w","errorCode":null,"errorMessage":"failed to verify storage permissions: %w","messagePattern":"failed to verify storage permissions: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"usecases/modulecomponents/usage/base_module.go","lineNumber":156,"sourceCode":"\tif b.config.Usage.ShardConcurrency != nil {\n\t\tif shardConcurrency := b.config.Usage.ShardConcurrency.Get(); shardConcurrency > 0 {\n\t\t\tb.shardConcurrency = shardConcurrency\n\t\t}\n\t}\n\t// push the parsed value in case the usage service was wired before Init\n\tif b.usageService != nil {\n\t\tb.usageService.SetShardConcurrency(b.shardConcurrency)\n\t}\n\n\t// Verify storage permissions (opt-in)\n\tvar shouldVerifyPermissions bool\n\tif b.config.Usage.VerifyPermissions != nil {\n\t\tshouldVerifyPermissions = b.config.Usage.VerifyPermissions.Get()\n\t}\n\n\tif shouldVerifyPermissions {\n\t\tif err := b.storage.VerifyPermissions(ctx); err != nil {\n\t\t\treturn fmt.Errorf(\"failed to verify storage permissions: %w\", err)\n\t\t}\n\t\tb.logger.Info(\"storage permissions verified successfully\")\n\t} else {\n\t\tb.logger.Info(\"storage permission verification skipped (disabled by configuration)\")\n\t}\n\n\t// try to adjust the initial interval, to avoid push gaps after Weaviate's restarts\n\tif err := b.adjustInitialInterval(config); err != nil {\n\t\tb.logger.Errorf(\"cannot adjust initial interval, falling back to: %v: %v\", b.interval, err)\n\t}\n\n\tb.logger.Infof(\"%s module initialized successfully\", b.moduleName)\n\treturn nil\n}\n\nfunc (b *BaseModule) collectAndUploadPeriodically(ctx context.Context) {\n\t// Validate intervals before creating tickers\n\tif b.interval <= 0 {","sourceCodeStart":138,"sourceCodeEnd":174,"githubUrl":"https://github.com/weaviate/weaviate/blob/75aa4b6d11f8818305aafd4440b4e32794f7ca04/usecases/modulecomponents/usage/base_module.go#L138-L174","documentation":"When config.Usage.VerifyPermissions is enabled, InitializeCommon calls storage.VerifyPermissions(ctx) at startup to confirm the configured storage backend is accessible and writable. Any underlying failure is wrapped with this prefix so the operator sees a permission/verification failure.","triggerScenarios":"Usage module with USE_USAGE_VERIFY_PERMISSIONS=true where the storage backend (e.g. object store) rejects the check due to bad credentials, missing bucket, or insufficient IAM permissions.","commonSituations":"Misconfigured cloud credentials (expired keys, wrong role/policy); bucket name typos; network/firewall blocking storage endpoint at boot.","solutions":["Inspect the wrapped cause (%w chain) for the underlying storage error and fix credentials/bucket/permissions","Verify IAM policy grants the required read/write actions on the storage backend","Confirm bucket/container names and region in the storage config","If verification is intentionally not wanted, disable it via the VerifyPermissions config option"],"exampleFix":"// before\naws s3 ls s3://my-bucket  # AccessDenied — IAM lacks s3:ListBucket\n// after\n# attach policy granting s3:ListBucket and s3:GetObject/PutObject on my-bucket, then restart","handlingStrategy":"try-catch","validationCode":"// pre-flight: verify storage credentials before starting\nif _, err := store.Stat(ctx, probePath); err != nil {\n  return fmt.Errorf(\"storage probe failed before init: %w\", err)\n}","typeGuard":null,"tryCatchPattern":"if err := module.Init(ctx, cfg, logger, metrics); err != nil {\n  var inner error\n  if errors.As(err, &inner) || errors.Unwrap(err) != nil {\n    logger.Fatalf(\"usage init failed: %v (cause: %v)\", err, errors.Unwrap(err))\n  }\n}","preventionTips":["Verify storage credentials and IAM policies in CI before deploying","Test bucket/container access with the same role the server runs as","Read the wrapped cause chain — this error always wraps the real storage failure","Consider a pre-flight permissions probe at deploy time rather than at boot"],"tags":["storage","permissions","startup","usage"],"backgroundTag":"storage-permission-denied","analyzedSha":"75aa4b6d11f8818305aafd4440b4e32794f7ca04","analyzedAt":"2026-09-04T14:58:20.392Z","contentChangedAt":"2026-09-04T14:58:20.392Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}