hashicorp/nomad · error

unable to update job submission: %v

Error message

unable to update job submission: %v

What it means

UpsertJob failed while persisting the job's submission metadata (original HCL/variables) via updateJobSubmission into the job_submission table, used for jobs submitted with HCL2 variable files so the source can be audited later. The failure aborts the registration transaction.

Source

Thrown at nomad/state/state_store.go:1904

	if err := s.upsertJobVersion(index, job, txn); err != nil {
		return fmt.Errorf("unable to upsert job into job_version table: %v", err)
	}

	if err := s.updateJobScalingPolicies(index, job, txn); err != nil {
		return fmt.Errorf("unable to update job scaling policies: %v", err)
	}

	if err := s.updateJobRecommendations(index, txn, existingJob, job); err != nil {
		return fmt.Errorf("unable to update job recommendations: %v", err)
	}

	if err := s.updateJobCSIPlugins(index, job, existingJob, txn); err != nil {
		return fmt.Errorf("unable to update job csi plugins: %v", err)
	}

	if err := s.updateJobSubmission(index, sub, job.Namespace, job.ID, job.Version, txn); err != nil {
		return fmt.Errorf("unable to update job submission: %v", err)
	}

	if err := s.updatePreservedValues(job, existingJob, req); err != nil {
		return fmt.Errorf("unable to update preserved values: %v", err)
	}

	// Insert the job
	if err := txn.Insert("jobs", job); err != nil {
		return fmt.Errorf("job insert failed: %v", err)
	}
	if err := txn.Insert("index", &IndexEntry{"jobs", index}); err != nil {
		return fmt.Errorf("index update failed: %v", err)
	}

	return nil
}

// CheckIdempotencyToken finds all children of the parent job ID and checks to

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Check the wrapped cause in server logs
2. Retry the job submission
3. Re-submit the job without the submission payload if that unblocks you (loses source-audit history for that version)
4. Upgrade Nomad if reproducible

// Ensure submission payload is set consistently when submitting with variables
// (CLI does this automatically; API callers must populate it or omit it)
if jobSource != "" && job.Submission == nil {
  return fmt.Errorf("submission source provided without JobSubmission payload")
}

func isJobSubmissionError(err error) bool {
  return err != nil && strings.Contains(err.Error(), "job submission")
}

_, err := client.Jobs().Register(job, nil, nil, nil)
if isJobSubmissionError(err) {
  // retry; optionally re-submit without the Submission field to unblock
  job.Submission = nil
  _, err = client.Jobs().Register(job, nil, nil, nil)
}
return err

When it happens

Trigger: Registering a job that includes JobSubmission data (submitted via API with the submission field) where the job_submission table write fails in-transaction.

Common situations: Clusters on Nomad 1.1+ using 'nomad job run' with variables where state store inconsistencies or upgrade migrations break the job_submission table.

Related errors

• job_submission requires a namespace
• job_submission requires a jobID
• job submission lookup failed: %v
• error parsing: root should be an object
• cannot specify Accessor ID

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/5207c3198bc39df0. Report an issue: GitHub.