hashicorp/nomad · warning

failed to update job %q (%s) launch time: %v

Error message

failed to update job %q (%s) launch time: %v

What it means

When the job was already tracked, Add updates its position in the launch-time heap via p.heap.Update(job, next). If the heap update fails (job no longer present or heap invariant issues), the job ID, namespace, and error are wrapped and returned.

Source

Thrown at nomad/periodic.go:223

	_, tracked := p.tracked[tuple]
	if disabled {
		if tracked {
			p.removeLocked(tuple)
		}

		// If the job is disabled and we aren't tracking it, do nothing.
		return nil
	}

	// Add or update the job.
	p.tracked[tuple] = job
	next, err := job.Periodic.Next(time.Now().In(job.Periodic.GetLocation()))
	if err != nil {
		return fmt.Errorf("failed adding job %s: %v", job.NamespacedID(), err)
	}
	if tracked {
		if err := p.heap.Update(job, next); err != nil {
			return fmt.Errorf("failed to update job %q (%s) launch time: %v", job.ID, job.Namespace, err)
		}
		p.logger.Debug("updated periodic job", "job", job.NamespacedID())
	} else {
		if err := p.heap.Push(job, next); err != nil {
			return fmt.Errorf("failed to add job %v: %v", job.ID, err)
		}
		p.logger.Debug("registered periodic job", "job", job.NamespacedID())
	}

	// Signal an update.
	select {
	case p.updateCh <- struct{}{}:
	default:
	}

	return nil
}

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Retry the job registration — the race is usually transient
2. Check for concurrent deregistration of the same periodic job in logs
3. Serialize job updates through the leader's state apply path (default) rather than direct dispatcher access
4. If it recurs at startup, inspect restorePeriodicDispatcher ordering and leader election churn

err := client.Jobs().Register(job, nil, nil)
if err != nil && strings.Contains(err.Error(), "failed to update job") {
    // transient heap race: brief wait and retry registration
    time.Sleep(time.Second); err = client.Jobs().Register(job, nil, nil)
}

When it happens

Trigger: p.heap.Update fails for an already-tracked job — typically a race where the job was removed from the heap concurrently (deregistration, periodic dispatch removal) between the tracked check and the Update call.

Common situations: Job deregistered by another request while an upsert for the same job is being applied; concurrent state-store applies racing the periodic dispatcher during leader handoff.

Related errors

• heap doesn't contain object with ID %q (%s)
• failed adding job %s: %v
• volume is currently unschedulable
• Missing job datacenters
• Job datacenter must be non-empty string

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/53ac00179ff208dd. Report an issue: GitHub.