vitessio/vitess · error

uncaught panic: %v from: %v

Error message

uncaught panic: %v from: %v

What it means

VtctldServer wraps each RPC handler with panicHandler, converting any recovered panic into a structured error 'uncaught panic: %v from: %v' including the stack trace. It exists so a bug in a vtctld RPC implementation becomes an RPC error instead of crashing the whole vtctld process.

Source

Thrown at go/vt/vtctl/grpcvtctldserver/server.go:127

		ts:  ts,
		tmc: tmc,
		ws:  workflow.NewServer(env, ts, tmc),
	}
}

// NewTestVtctldServer returns a new VtctldServer for the given topo server
// AND tmclient for use in tests. This should NOT be used in production.
func NewTestVtctldServer(ts *topo.Server, tmc tmclient.TabletManagerClient) *VtctldServer {
	return &VtctldServer{
		ts:  ts,
		tmc: tmc,
		ws:  workflow.NewServer(vtenv.NewTestEnv(), ts, tmc),
	}
}

func panicHandler(err *error) {
	if x := recover(); x != nil {
		*err = fmt.Errorf("uncaught panic: %v from: %v", x, string(debug.Stack()))
	}
}

// AddCellInfo is part of the vtctlservicepb.VtctldServer interface.
func (s *VtctldServer) AddCellInfo(ctx context.Context, req *vtctldatapb.AddCellInfoRequest) (resp *vtctldatapb.AddCellInfoResponse, err error) {
	span, ctx := trace.NewSpan(ctx, "VtctldServer.AddCellInfo")
	defer span.Finish()

	defer panicHandler(&err)

	if req.CellInfo.Root == "" {
		err = vterrors.Errorf(vtrpcpb.Code_FAILED_PRECONDITION, "CellInfo.Root must be non-empty")
		return nil, err
	}

	span.Annotate("cell", req.Name)
	span.Annotate("cell_root", req.CellInfo.Root)
	span.Annotate("cell_address", req.CellInfo.ServerAddress)

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Read the embedded stack trace to find the panicking function and fix/upgrade the code.
2. Validate the request fields before calling (non-nil cell info, valid keyspace/table names).
3. Reproduce with a minimal request and file/check a Vitess bug if it's a library defect.
4. Retry after correcting input or topo state that triggered the nil dereference.

Example fix

// before
resp, err := client.AddCellInfo(ctx, &vtctldatapb.AddCellInfoRequest{}) // nil CellInfo panics server-side
// after
resp, err := client.AddCellInfo(ctx, &vtctldatapb.AddCellInfoRequest{
    Name: "zone1",
    CellInfo: &topodatapb.CellInfo{ServerAddress: "etcd:2379", Root: "/vitess"},
})

// client-side: validate required request fields before sending
if req.GetName() == "" || req.GetCellInfo() == nil {
    return errors.New("AddCellInfo requires name and cellInfo")
}

func isPanicError(err error) bool {
    return err != nil && strings.Contains(err.Error(), "uncaught panic:")
}

resp, err := client.AddCellInfo(ctx, req)
if err != nil {
    if isPanicError(err) {
        // log full server stack from err and report/fix the request triggering it
        log.Error("vtctld panicked", slog.Any("error", err))
    }
    return err
}

When it happens

Trigger: Any vtctld RPC (AddCellInfo, AddCellsAlias, ApplyRoutingRules, ApplySchema, ApplyVSchema, Backup, etc.) whose implementation panics — nil pointer dereference on malformed requests, index out of range on empty topo results, assertion failures in called code.

Common situations: Sending requests with required fields unset (nil cell/topo options); topo corruption causing nil returns handled unsafely; bugs in a Vitess version triggered by unusual input — the panic message and stack point at the real cause.

Related errors

• GetKeyspaces(cluster = %s): %w
• failed to transition from state %s
• invalid key:value pair
• parse error
• failed to load static auth plugin. Plugin configured but grp

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/ab13d92b6ee0da4c. Report an issue: GitHub.