temporalio/temporal · critical

config.ESProcessorAckTimeout is nil

Error message

config.ESProcessorAckTimeout is nil

What it means

VisibilityStore.checkProcessor also panics when processorAckTimeout (derived from config.ESProcessorAckTimeout) is nil, even if the processor itself exists. AddBulkRequestAndWait uses this duration to wait for the indexer to acknowledge flushed bulk requests; without it the store cannot bound the wait, so the nil config is treated as broken history/service setup and panics.

Source

Thrown at common/persistence/visibility/store/elasticsearch/visibility_store.go:376

	}

	if !ack {
		// Returns retryable Unavailable error here because NACK from bulk processor
		// means that this request wasn't processed successfully and needs to be retried.
		// Visibility task processor retries all errors anyway, therefore, new request will be generated for the same visibility task.
		return serviceerror.NewUnavailable("visibility task received NACK")
	}
	return nil
}

func (s *VisibilityStore) checkProcessor() {
	if s.processor == nil {
		// must be a bug, check history setup
		panic("Elasticsearch processor is nil")
	}
	if s.processorAckTimeout == nil {
		// must be a bug, check history setup
		panic("config.ESProcessorAckTimeout is nil")
	}
}

func (s *VisibilityStore) ListWorkflowExecutions(
	ctx context.Context,
	request *manager.ListWorkflowExecutionsRequestV2,
) (*store.InternalListExecutionsResponse, error) {
	p, err := s.BuildSearchParametersV2(request, s.GetListFieldSorter)
	if err != nil {
		return nil, err
	}

	searchResult, err := s.esClient.Search(ctx, p)
	if err != nil {
		return nil, ConvertElasticsearchClientError("ListWorkflowExecutions failed", err, s.logger)
	}

	return s.GetListWorkflowExecutionsResponse(searchResult, request.Namespace, request.PageSize, nil)

View on GitHub (pinned to bde624efd1)

Solutions

1. Set esProcessor.ackTimeout (config.ESProcessorAckTimeout) in the visibility Elasticsearch config block, or apply the documented default during config load.
2. Ensure the store constructor normalizes nil config values to defaults instead of passing nil through (fix in NewVisibilityStore wiring).
3. Validate ES processor config at service startup before opening visibility traffic.
4. Fix test setup to pass a populated config struct with AckTimeout set.

Example fix

// before
store := elasticsearch.NewVisibilityStore(client, schema, index, logger, metrics, cfg, processorCfg{ /* AckTimeout unset */ })
// after
if cfg.ESProcessorAckTimeout == nil {
    cfg.ESProcessorAckTimeout = defaultESProcessorAckTimeout // e.g. dynamicconfig duration setting
}
store := elasticsearch.NewVisibilityStore(client, schema, index, logger, metrics, cfg, cfg.ESProcessorAckTimeout)

if cfg.ESProcessorAckTimeout == nil {
    return errors.New("elasticsearch visibility config missing esProcessor.ackTimeout")
}

func ackTimeoutConfigured(d *time.Duration) bool { return d != nil && *d > 0 }

// validate config at startup before the store serves traffic
if !ackTimeoutConfigured(cfg.ESProcessorAckTimeout) { return nil, fmt.Errorf("ESProcessorAckTimeout must be set") }
store := elasticsearch.NewVisibilityStore(...)

When it happens

Trigger: Calling AddBulkRequestAndWait when the store was created without an ESProcessorAckTimeout config value — the constructor skipped parsing/validating the config, or the config struct field was never populated during wiring.

Common situations: Elasticsearch visibility config block missing or partially filled (processor settings absent) while ES visibility is enabled; YAML config typo so ESProcessorAckTimeout defaults to nil and is never defaulted at load; test constructors passing zero-value config.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• requires a StartTime or CloseTime
• Elasticsearch processor is nil
• Unknown field type: %v
• unable to create AWS HTTP client for Elasticsearch: %w
• unable to create Elasticsearch client (URL = %v, username =

AI-assisted analysis of temporalio/temporal@bde624efd1 (2026-09-01). Data as JSON: /api/errors/a795fd5f25ac7f95. Report an issue: GitHub.