{"record":{"id":"118fd4c5314dc59b","repo":"weaviate/weaviate","slug":"init-shard-q-metrics-w","errorCode":null,"errorMessage":"init shard %q metrics: %w","messagePattern":"init shard %q metrics: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"adapters/repos/db/shard_init.go","lineNumber":66,"sourceCode":"\t\t\"shard\":  shardName,\n\t\t\"index\":  index.ID(),\n\t}).Debugf(\"initializing shard %q\", shardName)\n\n\tif err := shardusage.RemoveComputedUsageDataForUnloadedShard(index.path(), shardName); err != nil {\n\t\treturn nil, fmt.Errorf(\"shard %q: remove computed usage file for unloaded shard: %w\", shardName, err)\n\t}\n\n\tif err := newPropertyDeleteIndexHelper().ensureBucketsAreRemovedForNonExistentPropertyIndexes(index.path(), shardName, class); err != nil {\n\t\treturn nil, fmt.Errorf(\"shard %q: remove nonexistent property index buckets: %w\", shardName, err)\n\t}\n\n\tif err := newVectorDropIndexHelper().ensureFilesAreRemovedForDroppedVectorIndexes(index.path(), shardName, class); err != nil {\n\t\treturn nil, fmt.Errorf(\"shard %q: remove dropped vector index files: %w\", shardName, err)\n\t}\n\n\tmetrics, err := NewMetrics(index.logger, promMetrics, string(index.Config.ClassName), shardName)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"init shard %q metrics: %w\", shardName, err)\n\t}\n\tif index.Config.LazySegmentsDisabled {\n\t\tlazyLoadSegments = false // disabled globally\n\t}\n\n\tshutCtx, shutCtxCancel := context.WithCancelCause(context.Background())\n\n\ts := &Shard{\n\t\tindex:       index,\n\t\tclass:       class,\n\t\tname:        shardName,\n\t\tpromMetrics: promMetrics,\n\t\tmetrics:     metrics,\n\t\tslowQueryReporter: helpers.NewSlowQueryReporter(index.Config.QuerySlowLogEnabled,\n\t\t\tindex.Config.QuerySlowLogThreshold, index.logger),\n\t\treplicationMap:   pendingReplicaTasks{Tasks: make(map[string]replicaTask, 32)},\n\t\tcentralJobQueue:  jobQueueCh,\n\t\tscheduler:        scheduler,","sourceCodeStart":48,"sourceCodeEnd":84,"githubUrl":"https://github.com/weaviate/weaviate/blob/75aa4b6d11f8818305aafd4440b4e32794f7ca04/adapters/repos/db/shard_init.go#L48-L84","documentation":"Weaviate wraps the failure of NewMetrics() during shard creation in NewShard (adapters/repos/db/shard_init.go:66). NewMetrics registers shard-level Prometheus metrics (counters/gauges/histograms labeled by class and shard) with the provided registerer. If registration fails — typically because a metric with the same fully-qualified name is already registered — shard startup aborts and the shard is not created.","triggerScenarios":"NewShard is called and NewMetrics returns an error, most commonly prometheus.AlreadyRegisteredError from duplicate metric registration: creating two shards whose metric name components (class name/shard name) collide after sanitization, re-registering on shard re-creation (e.g. tenant offloading/re-onloading) without an unregister, or passing an already-populated registerer.","commonSituations":"Operators see this when starting Weaviate with a PROMETHEUS_MONITORING_ENABLED setup where a previous registration was not cleaned up, when class/shard names sanitize to the same Prometheus label-safe string, or in tests that reuse a single prometheus.Registerer across multiple NewShard calls. Also possible after offload/onload cycles if cleanup of the old shard's metrics failed.","solutions":["Check the wrapped error for prometheus.AlreadyRegisteredError and ensure the previous shard's metrics are unregistered before re-creating it (complete offload/cleanup before onload)","Verify class and shard names don't collide after metric-name sanitization; rename the offending shard or class","If embedding Weaviate or running tests, pass a fresh prometheus.NewRegistry() per shard instead of reusing one registerer","Check disk/permissions are not the cause — inspect the full wrapped chain for the underlying NewMetrics error"],"exampleFix":"// before (test): reuse one registry for two shards\nreg := prometheus.NewRegistry()\ns1, _ := db.NewShard(ctx, monitoring.NewPrometheusMetrics(reg), \"shard-1\", idx, class, ...)\ns2, _ := db.NewShard(ctx, monitoring.NewPrometheusMetrics(reg), \"shard-1\", idx2, class, ...) // duplicate registration\n// after\ns2, _ := db.NewShard(ctx, monitoring.NewPrometheusMetrics(prometheus.NewRegistry()), \"shard-1\", idx2, class, ...)","handlingStrategy":"validation","validationCode":"// Before creating a shard with a custom registerer, ensure no metric of the same\n// name is registered. In tests/embedding scenarios:\nfunc canRegister(reg prometheus.Registerer, name string) bool {\n\tcollector := prometheus.NewCounter(prometheus.CounterOpts{Name: name})\n\tif err := reg.Register(collector); err != nil {\n\t\treg.Unregister(collector) // roll back the probe\n\t\treturn false\n\t}\n\treg.Unregister(collector)\n\treturn true\n}","typeGuard":null,"tryCatchPattern":"shard, err := db.NewShard(ctx, promMetrics, shardName, index, class, ...)\nif err != nil {\n\tvar are prometheus.AlreadyRegisteredError\n\tif errors.As(err, &are) {\n\t\t// duplicate metric registration: clean up old registration and retry once\n\t}\n\treturn fmt.Errorf(\"shard creation failed: %w\", err)\n}","preventionTips":["Always unregister a shard's metrics during offload/delete before the shard can be re-created","In tests, create a fresh prometheus.NewRegistry() per shard instead of sharing one registerer","Keep class and shard names distinct enough to survive metric-name sanitization","Alert on this log message early at startup — it fails fast and deterministically"],"tags":["shard-init","prometheus","metrics","startup"],"backgroundTag":"duplicate-metric-registration","analyzedSha":"75aa4b6d11f8818305aafd4440b4e32794f7ca04","analyzedAt":"2026-09-04T14:58:20.392Z","contentChangedAt":"2026-09-04T14:58:20.392Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}