What causes "error creating growable tracker: %v" in apache/beam?

Fix the NATS URI/credentials so natsFn.Setup succeeds and fn.js is non-nil before tracker creation.

error creating growable tracker

Error message

error creating growable tracker: %v

What it means

Returned by readFn.createRTracker (used by RestrictionSize and CreateTracker) when offsetrange.NewGrowableTracker fails to construct a growable restriction tracker for an unbounded restriction (End == math.MaxInt64). The growable tracker needs an end-estimator callback into JetStream; construction fails if the estimator cannot be set up, typically because the JetStream context (fn.js) is nil or the stream/subject is invalid.

Solutions

Fix the NATS URI/credentials so natsFn.Setup succeeds and fn.js is non-nil before tracker creation.
Verify the stream name passed to natsio.Read exists (nats stream list); an unknown stream breaks the end estimator.
Confirm the subject filter matches at least the stream's subjects (nats stream info <stream> subjects).
If you only need bounded reads, set the EndSeqNo option below math.MaxInt64 so a plain static tracker is used and no estimator is needed.

Example fix

// before
pc := natsio.Read(s, uri, stream, subject)
// after
pc := natsio.Read(s, uri, stream, subject,
  natsio.EndSeqNo(1000)) // bounded: avoids growable tracker + end estimator entirely

Defensive patterns

Strategy: validation

Validate before calling

// validate before Read()
js, err := jetstream.New(nc)
if err != nil { return err }
if _, err := js.StreamInfo(ctx, stream); err != nil {
    return fmt.Errorf("cannot resolve stream %q for end estimator: %w", stream, err)
}
if subject == "" { return errors.New("subject must be non-empty") }

Try / catch

rt, err := offsetrange.NewGrowableTracker(rest, estimator)
if err != nil {
    // fall back to a bounded static tracker for a conservative window
    return offsetrange.NewTracker(offsetrange.Restriction{Start: rest.Start, End: rest.Start + 1_000_000}), nil
}

Prevention

Ensure the NATS connection succeeds at Setup before unbounded reads (check URI/creds)
Validate stream and subject names against `nats stream info` before launching
Set an explicit EndSeqNo when you know the data range to avoid the growable tracker entirely
Keep one shared JetStream context instance so fn.js is never nil

When it happens

Trigger: natsio.Read called with EndSeqNo left at default math.MaxInt64 AND fn.js is nil (Setup/connection failed earlier) or newEndEstimator's stream/subject references don't resolve, so offsetrange.NewGrowableTracker returns an error at read.go:248.

Common situations: Passing an invalid URI or bad credentials so the NATS connection/JetStream context was never established; typos in the stream name used both for the consumer and the end estimator; calling RestrictionSize before Setup on a worker where the connection never initialized.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/259e3510896b8917. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/io/natsio/read.go:250

			return sdf.StopProcessing(), fmt.Errorf("error in message batch: %v", err)
		}

		if count == 0 {
			fn.updateWatermarkManually(we)
			return sdf.ResumeProcessingIn(resumeDelay), nil
		}
	}
}

func (fn *readFn) createRTracker(rest offsetrange.Restriction) (sdf.RTracker, error) {
	if rest.End < math.MaxInt64 {
		return offsetrange.NewTracker(rest), nil
	}

	estimator := newEndEstimator(fn.js, fn.Stream, fn.Subject)
	rt, err := offsetrange.NewGrowableTracker(rest, estimator)
	if err != nil {
		return nil, fmt.Errorf("error creating growable tracker: %v", err)
	}

	return rt, nil
}

func (fn *readFn) createConsumer(
	ctx context.Context,
	startSeqNo int64,
) (jetstream.Consumer, error) {
	cfg := jetstream.OrderedConsumerConfig{
		FilterSubjects:   []string{fn.Subject},
		DeliverPolicy:    jetstream.DeliverByStartSequencePolicy,
		OptStartSeq:      uint64(startSeqNo),
		MaxResetAttempts: 5,
	}

	cons, err := fn.js.OrderedConsumer(ctx, fn.Stream, cfg)
	if err != nil {

View on GitHub (pinned to 12126d8942)