apache/beam · error · ValueError

Per element sleep time must be at least 1e-3 after being…

Error message

Per element sleep time must be at least 1e-3 after being divided among output elements.

What it means

When a synthetic step fans out records (output_records_per_input_record > 1), the per-element delay is divided among output elements; __init__ raises ValueError if the divided delay falls below 1e-3 seconds, since the step cannot sleep that little. This is the fan-out-aware variant of the minimum-delay constraint.

Solutions

Increase per_element_delay_sec so that delay // output_records_per_input_record >= 0.001
Lower output_records_per_input_record
Move the remaining delay to per_bundle_delay_sec instead

Example fix

// before
SyntheticStep(per_element_delay_sec=0.002, output_records_per_input_record=5)
// after
SyntheticStep(per_element_delay_sec=0.005, output_records_per_input_record=5)

Defensive patterns

Strategy: validation

Validate before calling

if per_element_delay_sec and \
        per_element_delay_sec // output_records_per_input_record < 1e-3:
    raise ValueError('increase delay or reduce output_records_per_input_record')

Type guard

def delay_survives_fanout(delay, fanout):
    return not delay or delay // fanout >= 1e-3

Try / catch

try:
    step = SyntheticStep(per_element_delay_sec=d, output_records_per_input_record=n)
except ValueError:
    d = max(d, 1e-3 * n)
    step = SyntheticStep(per_element_delay_sec=d, output_records_per_input_record=n)

Prevention

Check the delay/fanout interaction whenever tuning either parameter
Derive delay from fanout programmatically instead of hardcoding
Validate specs at parse time

When it happens

Trigger: Constructing the step with per_element_delay_sec // output_records_per_input_record < 1e-3, e.g. delay 0.002 with 5 output records per input (0.002//5 = 0 < 1e-3).

Common situations: Large fan-out factors combined with small delays in load-test specs; tuning delay/fan-out independently without checking their interaction.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9c7f53ba33ea3341. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/testing/synthetic_pipeline.py:280

    disable_liquid_sharding=False,
    size_estimate_override=None,
):
  """A function which returns a SyntheticSDFStep with given parameters. """
  class SyntheticSDFStep(beam.DoFn):
    """A SplittableDoFn of which behavior can be controlled through prespecified
       parameters.
    """
    def __init__(
        self,
        per_element_delay_sec_arg,
        per_bundle_delay_sec_arg,
        output_filter_ratio_arg,
        output_records_per_input_record_arg):
      if per_element_delay_sec_arg:
        per_element_delay_sec_arg = (
            per_element_delay_sec_arg // output_records_per_input_record_arg)
        if per_element_delay_sec_arg < 1e-3:
          raise ValueError(
              'Per element sleep time must be at least 1e-3 after being '
              'divided among output elements.')
      self._per_element_delay_sec = per_element_delay_sec_arg
      self._per_bundle_delay_sec = per_bundle_delay_sec_arg
      self._output_filter_ratio = output_filter_ratio_arg

    def start_bundle(self):
      self._start_time = time.time()

    def finish_bundle(self):
      # The target is for the enclosing stage to take as close to as possible
      # the given number of seconds, so we only sleep enough to make up for
      # overheads not incurred elsewhere.
      to_sleep = self._per_bundle_delay_sec - (time.time() - self._start_time)

      # Ignoring sub-millisecond sleep times.
      if to_sleep >= 1e-3:
        time.sleep(to_sleep)

View on GitHub (pinned to 12126d8942)