hashicorp/nomad · error
periodic launch lookup failed: %v
Error message
periodic launch lookup failed: %v
What it means
UpsertPeriodicLaunch checks for an existing PeriodicLaunch row (per namespace+ID) with txn.First before writing the new launch record. This error wraps a First failure on the periodic_launch table's "id" index. It is an internal memdb error, distinct from simply not finding an existing launch (which returns nil).
Source
Thrown at nomad/state/state_store.go:3347
return structs.ErrCSIPluginInUse
}
err = txn.Delete(TableCSIPlugins, plug)
if err != nil {
return fmt.Errorf("csi_plugins delete error: %v", err)
}
return txn.Commit()
}
// UpsertPeriodicLaunch is used to register a launch or update it.
func (s *StateStore) UpsertPeriodicLaunch(index uint64, launch *structs.PeriodicLaunch) error {
txn := s.db.WriteTxn(index)
defer txn.Abort()
// Check if the job already exists
existing, err := txn.First("periodic_launch", "id", launch.Namespace, launch.ID)
if err != nil {
return fmt.Errorf("periodic launch lookup failed: %v", err)
}
// Setup the indexes correctly
if existing != nil {
launch.CreateIndex = existing.(*structs.PeriodicLaunch).CreateIndex
launch.ModifyIndex = index
} else {
launch.CreateIndex = index
launch.ModifyIndex = index
}
// Insert the job
if err := txn.Insert("periodic_launch", launch); err != nil {
return fmt.Errorf("launch insert failed: %v", err)
}
if err := txn.Insert("index", &IndexEntry{"periodic_launch", index}); err != nil {
return fmt.Errorf("index update failed: %v", err)
}View on GitHub (pinned to 482b49bf1a)
Solutions
- Read the wrapped error for the underlying memdb cause.
- Restart the server agent; leadership will re-establish and rebuild state from Raft.
- Align Nomad versions across all servers.
- If it persists, run the Raft recovery procedure.
Defensive patterns
Strategy: retry
Try / catch
err := store.UpsertPeriodicLaunch(index, launch)
if err != nil && strings.Contains(err.Error(), "periodic launch lookup failed") {
// leadership will retry on the next periodic tick
return retry.WithBackoff(3, func() error { return store.UpsertPeriodicLaunch(index, launch) })
} Prevention
- Rely on the periodic runner's next tick to retry failed launches
- Restart servers promptly if lookup errors repeat (state rebuild)
- Keep Nomad versions consistent across servers
- Monitor leader memory pressure
When it happens
Trigger: Job submission of a periodic job where the leader calls UpsertPeriodicLaunch and txn.First("periodic_launch", "id", namespace, id) errors — memdb index/table failure.
Common situations: Corrupt state store on the leader, version mismatch after upgrade, or heavy memory pressure corrupting memdb during Raft replay.
Related errors
- launch insert failed: %v
- launch not found
- error parsing: root should be an object
- cannot specify Accessor ID
- network already configured but not found in state
AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04).
Data as JSON: /api/errors/3d41681d94840e10.
Report an issue: GitHub.