{"record":{"id":"3b23c8a7bd9950a0","repo":"vitessio/vitess","slug":"failed-to-read-tablet-q-from-vtorc-db-w","errorCode":null,"errorMessage":"failed to read tablet %q from vtorc db: %w","messagePattern":"failed to read tablet %q from vtorc db: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"go/vt/vtorc/logic/topology_recovery.go","lineNumber":461,"sourceCode":"\tif !isERSEnabled(analysisEntry) {\n\t\tlog.Warn(\"VTOrc not configured to run EmergencyReparentShard, skipping recovering \" + recoveryName)\n\t\treturn recoveryAttempted, topologyRecovery, nil\n\t}\n\treturn runEmergencyReparentOp(ctx, analysisEntry, recoveryName, false, logger)\n}\n\n// isPrimaryReachable loads the analyzed primary tablet from the vtorc DB\n// and checks whether tabletmanager Ping is reachable within a short timeout.\n// It returns false with a nil error when the tablet is missing or incomplete so\n// the caller can decide whether to skip recovery without failing outright.\nfunc isPrimaryReachable(ctx context.Context, analysisEntry *inst.DetectionAnalysis) (bool, error) {\n\tif analysisEntry == nil || analysisEntry.AnalyzedInstanceAlias == nil {\n\t\treturn false, nil\n\t}\n\n\ttablet, err := inst.ReadTablet(analysisEntry.AnalyzedInstanceAlias)\n\tif err != nil {\n\t\treturn false, fmt.Errorf(\"failed to read tablet %q from vtorc db: %w\", topoproto.TabletAliasString(analysisEntry.AnalyzedInstanceAlias), err)\n\t}\n\n\tif tablet == nil || tablet.Hostname == \"\" || tablet.PortMap == nil {\n\t\treturn false, nil\n\t}\n\n\tgrpcPort, ok := tablet.PortMap[\"grpc\"]\n\tif !ok || grpcPort == 0 {\n\t\treturn false, nil\n\t}\n\n\tif tmc == nil {\n\t\treturn false, errors.New(\"tablet manager client is not initialized\")\n\t}\n\n\tctx, cancel := context.WithTimeout(ctx, 2*time.Second)\n\tdefer cancel()\n","sourceCodeStart":443,"sourceCodeEnd":479,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/vtorc/logic/topology_recovery.go#L443-L479","documentation":"During primary recovery, isPrimaryReachable checks whether the analyzed (incapacitated) primary tablet is actually reachable by first reading its Tablet record from VTOrc's local backing database (inst.ReadTablet). If that DB read fails with a non-nil error, the reachability check is aborted and this wrapped error is returned, preventing the recovery from making a decision.","triggerScenarios":"recoverIncapacitatedPrimary calls isPrimaryReachable for an analysis entry with a non-nil AnalyzedInstanceAlias, and inst.ReadTablet(alias) returns a DB error (backend unavailable, corrupted vtorc DB, query failure).","commonSituations":"VTOrc's MySQL backend is down or restarting during a failover; network partition between VTOrc and its metadata store; table schema drift after a version upgrade; disk-full on the backend.","solutions":["Check that VTOrc's backend database is up and reachable; restart VTOrc once connectivity is restored","Inspect VTOrc logs for the underlying wrapped error (%w) to identify the DB failure","Verify the inst.tablets table exists and matches the expected schema for your Vitess version (run any pending migrations)","Re-run recovery; the analysis loop will re-detect the problem and retry the read"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Check vtorc backend reachability first\nif err := db.PingContext(ctx); err != nil { return err }","typeGuard":null,"tryCatchPattern":"tablet, err := inst.ReadTablet(alias)\nif err != nil {\n    var retryable bool\n    if errors.Is(err, sql.ErrConnDone) || errors.Is(err, driver.ErrBadConn) { retryable = true }\n    return fmt.Errorf(\"failed to read tablet %q: %w (retryable=%v)\", aliasStr, err, retryable)\n}","preventionTips":["Monitor vtorc backend DB health","Keep vtorc schema migrations current","Alert on DB connection pool saturation"],"tags":["vtorc","database","topology-recovery"],"backgroundTag":"tablet-read-failed","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}