{"record":{"id":"33ca8557d13f7e49","repo":"vitessio/vitess","slug":"getworkflows-keyspace-s-active-only-v-fail-33ca85","errorCode":null,"errorMessage":"GetWorkflows(keyspace = %s, active_only = %v) failed: %w","messagePattern":"GetWorkflows\\(keyspace = (.+?), active_only = (.+?)\\) failed: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"go/vt/vtadmin/cluster/cluster.go","lineNumber":807,"sourceCode":"\t\t\tspan.Annotate(\"keyspace\", ks)\n\t\t\tspan.Annotate(\"active_only\", opts.ActiveOnly)\n\n\t\t\tif err := c.workflowReadPool.Acquire(ctx); err != nil {\n\t\t\t\terr = fmt.Errorf(\"GetWorkflows(keyspace = %s, active_only = %v) failed to acquire workflowReadPool: %w\", ks, opts.ActiveOnly, err)\n\t\t\t\trec.RecordError(err)\n\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tresp, err := c.Vtctld.GetWorkflows(ctx, &vtctldatapb.GetWorkflowsRequest{\n\t\t\t\tKeyspace:    ks,\n\t\t\t\tActiveOnly:  opts.ActiveOnly,\n\t\t\t\tIncludeLogs: true,\n\t\t\t})\n\t\t\tc.workflowReadPool.Release()\n\n\t\t\tif err != nil {\n\t\t\t\terr = fmt.Errorf(\"GetWorkflows(keyspace = %s, active_only = %v) failed: %w\", ks, opts.ActiveOnly, err)\n\t\t\t\trec.RecordError(err)\n\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tworkflows := make([]*vtadminpb.Workflow, 0, len(resp.Workflows))\n\t\t\tfor _, wf := range resp.Workflows {\n\t\t\t\tworkflow := &vtadminpb.Workflow{\n\t\t\t\t\tCluster:  clusterpb,\n\t\t\t\t\tKeyspace: ks,\n\t\t\t\t\tWorkflow: wf,\n\t\t\t\t}\n\n\t\t\t\tif opts.Filter(workflow) {\n\t\t\t\t\tworkflows = append(workflows, workflow)\n\t\t\t\t}\n\t\t\t}\n","sourceCodeStart":789,"sourceCodeEnd":825,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/vtadmin/cluster/cluster.go#L789-L825","documentation":"After acquiring a workflowReadPool slot, the per-keyspace GetWorkflows vtctld RPC itself failed. The error is wrapped with the keyspace and active_only flag so operators know exactly which keyspace's workflow query broke. Other keyspaces may still succeed because each goroutine fails independently.","triggerScenarios":"Per-keyspace c.Vtctld.GetWorkflows RPC returning an error — vtctld unavailable, keyspace missing from topology mid-flight, or RPC error while including workflow logs.","commonSituations":"Keyspace being deleted/resharded concurrently with the query; vtctld error reading VReplication workflow state; transient network blip; ctx deadline during a large IncludeLogs fetch.","solutions":["Inspect the wrapped %w cause for the vtctld-side error","Retry the failed keyspace's query","Confirm the keyspace still exists in the topology","Check vtctld logs for VReplication/topo errors for that keyspace"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// verify keyspace exists before querying workflows\nresp, err := vtctldClient.GetKeyspaces(ctx, &vtctldatapb.GetKeyspacesRequest{})\nif err != nil { return err }\nvar found bool\nfor _, ks := range resp.Keyspaces { if ks.Name == keyspace { found = true } }\nif !found { return fmt.Errorf(\"keyspace %s does not exist\", keyspace) }","typeGuard":null,"tryCatchPattern":"wfs, err := cluster.GetWorkflows(ctx, keyspace, opts)\nif err != nil {\n    var perKS string\n    if errors.As(err, &perKS) || strings.Contains(err.Error(), keyspace) {\n        // retry just this keyspace\n    }\n    return err\n}","preventionTips":["Check keyspace existence before workflow queries","Monitor vtctld VReplication errors","Avoid querying keyspaces mid-reshard/deletion","Set sane timeouts for IncludeLogs requests"],"tags":["vtadmin","vtctld","vreplication","rpc"],"backgroundTag":"vtctld-rpc-failed","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}