What causes "failed to add job %v: %v" in hashicorp/nomad?

Update the existing job instead of adding it (the Add path takes the update branch when the job is tracked — ensure you pass the current job object so the tracked lookup succeeds).

hashicorp/nomad · error

failed to add job %v: %v

Error message

failed to add job %v: %v

What it means

This error is returned by PeriodicDispatch.Add when pushing a new periodic job onto the internal periodic heap fails. The heap Push only fails when a job with the same ID and namespace is already tracked, so this signals a duplicate registration attempt for a periodic job.

Source

Thrown at nomad/periodic.go:228

		// If the job is disabled and we aren't tracking it, do nothing.
		return nil
	}

	// Add or update the job.
	p.tracked[tuple] = job
	next, err := job.Periodic.Next(time.Now().In(job.Periodic.GetLocation()))
	if err != nil {
		return fmt.Errorf("failed adding job %s: %v", job.NamespacedID(), err)
	}
	if tracked {
		if err := p.heap.Update(job, next); err != nil {
			return fmt.Errorf("failed to update job %q (%s) launch time: %v", job.ID, job.Namespace, err)
		}
		p.logger.Debug("updated periodic job", "job", job.NamespacedID())
	} else {
		if err := p.heap.Push(job, next); err != nil {
			return fmt.Errorf("failed to add job %v: %v", job.ID, err)
		}
		p.logger.Debug("registered periodic job", "job", job.NamespacedID())
	}

	// Signal an update.
	select {
	case p.updateCh <- struct{}{}:
	default:
	}

	return nil
}

// Remove stops tracking the passed job. If the job is not tracked, it is a
// no-op.
func (p *PeriodicDispatch) Remove(namespace, jobID string) error {
	p.l.Lock()
	defer p.l.Unlock()

View on GitHub (pinned to 482b49bf1a)

Solutions

Update the existing job instead of adding it (the Add path takes the update branch when the job is tracked — ensure you pass the current job object so the tracked lookup succeeds).
Remove the existing job via PeriodicDispatch.Remove before calling Add again.
Check for duplicate job IDs/namespaces in your job submissions.
If seen during restore, verify state store restore logic doesn't re-add already-tracked jobs.

Example fix

// before
p.heap.Push(job, next) // panics/errors if already tracked
// after
if _, tracked := p.tracked[tuple]; tracked {
    if err := p.heap.Update(job, next); err != nil { return err }
} else if err := p.heap.Push(job, next); err != nil {
    return fmt.Errorf("failed to add job %v: %v", job.ID, err)
}

Defensive patterns

Strategy: validation

Validate before calling

// Go: check the job isn't already tracked before Add
if _, exists := dispatcher.tracked[job.NamespacedID()]; exists {
    // treat as update path or skip
}

Try / catch

// Go
if err := d.Add(job); err != nil {
    if strings.Contains(err.Error(), "failed to add job") {
        return d.Remove(job) // then re-add
    }
    return err
}

Prevention

Ensure job IDs are unique per namespace before submission
Route all periodic registrations through the update-aware Add path
Guard against duplicate upserts in applyUpsertJob

When it happens

Trigger: Calling Add (directly or via applyUpsertJob RPC or restorePeriodicDispatcher) for a periodic job whose NamespacedID (ID+Namespace) is already present in the dispatcher's heap, i.e. the heap wasn't updated/removed beforehand.

Common situations: Re-registering the same periodic job without deregistering it first; restoring state on leader election where the job is already in the heap; racing upserts of the same job ID in the same namespace.

Related errors

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/3f8de5ea9fcbbeac. Report an issue: GitHub.