hashicorp/nomad · error

job submissions lookup failed: %v

Error message

job submissions lookup failed: %v

What it means

GetJobSubmissions lists all job_submission table entries (HCL/spec submissions attached to job versions). The error wraps a failure from txn.Get on the job_submission table with the id index, meaning the table lookup itself failed rather than returning an iterator.

Source

Thrown at nomad/state/state_store.go:2298

	// Delete the oldest one
	d := all[max]
	if err := txn.Delete("job_version", d); err != nil {
		return fmt.Errorf("failed to delete job %v (%d) from job_version", d.ID, d.Version)
	}

	return nil
}

// GetJobSubmissions returns an iterator that contains all job submissions
// stored within state. This is not currently exposed via RPC and is only used
// for snapshot persist and restore functionality.
func (s *StateStore) GetJobSubmissions(ws memdb.WatchSet) (memdb.ResultIterator, error) {
	txn := s.db.ReadTxn()

	// Walk the entire table to get all job submissions.
	iter, err := txn.Get(TableJobSubmission, indexID)
	if err != nil {
		return nil, fmt.Errorf("job submissions lookup failed: %v", err)
	}
	ws.Add(iter.WatchCh())

	return iter, nil
}

// JobSubmission returns the original HCL/Variables context of a job, if available.
//
// Note: it is a normal case for the submission context to be unavailable, in which case
// nil is returned with no error.
func (s *StateStore) JobSubmission(ws memdb.WatchSet, namespace, jobName string, version uint64) (*structs.JobSubmission, error) {
	txn := s.db.ReadTxn()
	return s.jobSubmission(ws, namespace, jobName, version, txn)
}

func (s *StateStore) jobSubmission(ws memdb.WatchSet, namespace, jobName string, version uint64, txn Txn) (*structs.JobSubmission, error) {
	watchCh, existing, err := txn.FirstWatch("job_submission", "id", namespace, jobName, version)
	if err != nil {

View on GitHub (pinned to 482b49bf1a)

Solutions

  1. Check the wrapped %v cause for the memdb error
  2. Retry the request; read txns are lightweight and the failure is usually transient
  3. If persistent, verify state store integrity and file a Nomad issue
Defensive patterns

Strategy: try-catch

Type guard

func isLookupErr(err error) bool { return err != nil && strings.Contains(err.Error(), "job submissions lookup failed") }

Try / catch

iter, err := client.Jobs().Submissions(...)
if err != nil {
  log.Printf("submission list unavailable: %v", err) // inspect wrapped cause
  return fallbackResult
}

Prevention

When it happens

Trigger: Calling StateStore.GetJobSubmissions (or the JobSubmissionsList RPC) when the memdb lookup on TableJobSubmission/indexID fails, e.g. invalid index name or txn error.

Common situations: Rare internal errors; usually only encountered by developers extending Nomad's state store or during corrupted indexes.

Related errors


AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/0d67d28e972aa9f5. Report an issue: GitHub.