What causes "jsonnetsecure: acquire" in ory/hydra?

Check `errors.Is(err, jsonnetsecure.ErrProcessPoolClosed)` — if so, stop using the pool after Close or create a new one

jsonnetsecure: acquire

Error message

jsonnetsecure: acquire

What it means

`EvaluateAnonymousSnippet` fetches a worker from the puddle resource pool via `vm.pool.puddle.Acquire(ctx)`; any acquisition failure is wrapped as this error. Commonly this is the caller's context being cancelled/expired while waiting for a free worker, the pool being closed (ErrProcessPoolClosed), or worker construction (newWorker) failing while the pool tries to create resources.

Source

Thrown at oryx/jsonnetsecure/jsonnet_pool.go:266

func (vm *processPoolVM) EvaluateAnonymousSnippet(filename string, snippet string) (_ string, err error) {
	tracer := trace.SpanFromContext(vm.ctx).TracerProvider().Tracer("")
	ctx, span := tracer.Start(vm.ctx, "jsonnetsecure.processPoolVM.EvaluateAnonymousSnippet", trace.WithAttributes(attribute.String("filename", filename)))
	defer otelx.End(span, &err)

	params := vm.params
	params.Filename = filename
	params.Snippet = snippet
	pp, err := json.Marshal(params)
	if err != nil {
		return "", errors.Wrap(err, "jsonnetsecure: marshal")
	}

	ctx = context.WithValue(ctx, contextValuePath, vm.path)
	ctx = context.WithValue(ctx, contextValueArgs, vm.args)
	worker, err := vm.pool.puddle.Acquire(ctx)
	if err != nil {
		return "", errors.Wrap(err, "jsonnetsecure: acquire")
	}

	ctx, cancel := context.WithTimeoutCause(ctx, 1*time.Second, errors.Errorf("failed to run jsonnet within 1s: filename=%s", filename))
	defer cancel()
	result, err := worker.Value().eval(ctx, pp)
	if err != nil {
		worker.Destroy()
		return "", errors.Wrap(err, "jsonnetsecure: eval")
	} else {
		worker.Release()
	}

	if strings.HasPrefix(result, "ERROR: ") {
		return "", errors.New("jsonnetsecure: " + result)
	}

	return result, nil
}

View on GitHub (pinned to 4174065ffb)

Solutions

Check `errors.Is(err, jsonnetsecure.ErrProcessPoolClosed)` — if so, stop using the pool after Close or create a new one
Increase the pool size (NewProcessPool) to match concurrent evaluation demand
Verify the worker binary path and that newWorker succeeds (look for accompanying newWorker/warm-up errors in logs)
Check the wrapped context error: deadline/cancellation from upstream means either bigger pool or longer timeout

Example fix

// before: shared pool closed during shutdown while requests still evaluate
pool := jsonnetsecure.NewProcessPool(10)
defer pool.Close()
...requests may still call vm.EvaluateAnonymousSnippet after Close → "jsonnetsecure: acquire"
// after: drain work before closing
wg.Wait() // let in-flight evaluations finish
pool.Close()

Defensive patterns

Strategy: retry

Validate before calling

// before calling: ensure the pool is open and has capacity headroom
if pool.Stat() == nil {
    return errors.New("pool not initialized")
}
// stat := pool.Stat(); if stat.ConstructionWorkersCount==0 && stat.TotalResources()==0 && poolRecentlyClosed { ... }

Type guard

func isPoolClosed(err error) bool {
    return errors.Is(err, jsonnetsecure.ErrProcessPoolClosed)
}

Try / catch

result, err := vm.EvaluateAnonymousSnippet("f.jsonnet", snippet)
if err != nil && strings.Contains(err.Error(), "jsonnetsecure: acquire") {
    if errors.Is(err, jsonnetsecure.ErrProcessPoolClosed) {
        return "", err // do not retry a closed pool
    }
    select {
    case <-time.After(100 * time.Millisecond): // transient contention; retry once
        result, err = vm.EvaluateAnonymousSnippet("f.jsonnet", snippet)
    }
}

Prevention

Size the pool for peak concurrent evaluations (remember the internal minimum of 5)
Never call Pool.Close() while requests may still evaluate; drain first
Check upstream context deadlines — acquisition honors the caller's ctx
Monitor puddle stats (waiters, acquisitions) to detect saturation early

When it happens

Trigger: Calling EvaluateAnonymousSnippet when all workers are busy and the ctx expires before a worker is released; Pool.Close() was called (ErrProcessPoolClosed); underlying newWorker failures (exec, pipes, warm-up) surface through acquisition.

Common situations: Bursts of concurrent jsonnet evaluations exceeding pool size (min 5) with short-lived contexts; shutdown racing in-flight evaluations; the worker binary missing so pool refill keeps failing; context deadlines from upstream request timeouts.

Related errors

AI-assisted analysis of ory/hydra@4174065ffb (2026-09-03). Data as JSON: /api/errors/4c3539b95cbb58ba. Report an issue: GitHub.