{"record":{"id":"b65a7c3cfe8301d5","repo":"apache/beam","slug":"bucket-boundaries-requires-length-fn-to-be-set","errorCode":null,"errorMessage":"bucket_boundaries requires length_fn to be set.","messagePattern":"bucket_boundaries requires length_fn to be set\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/transforms/util.py","lineNumber":1087,"sourceCode":"  _DEFAULT_BUCKET_BOUNDARIES = [16, 32, 64, 128, 256, 512]\n\n  def __init__(\n      self,\n      min_batch_size=1,\n      max_batch_size=10000,\n      target_batch_overhead=.05,\n      target_batch_duration_secs=10,\n      target_batch_duration_secs_including_fixed_cost=None,\n      max_batch_duration_secs=None,\n      *,\n      element_size_fn=lambda x: 1,\n      variance=0.25,\n      clock=time.time,\n      record_metrics=True,\n      length_fn=None,\n      bucket_boundaries=None):\n    if bucket_boundaries is not None and length_fn is None:\n      raise ValueError('bucket_boundaries requires length_fn to be set.')\n    if bucket_boundaries is not None:\n      if (not bucket_boundaries or any(b <= 0 for b in bucket_boundaries) or\n          bucket_boundaries != sorted(bucket_boundaries)):\n        raise ValueError(\n            'bucket_boundaries must be a non-empty sorted list of '\n            'positive values.')\n    self._batch_size_estimator = _BatchSizeEstimator(\n        min_batch_size=min_batch_size,\n        max_batch_size=max_batch_size,\n        target_batch_overhead=target_batch_overhead,\n        target_batch_duration_secs=target_batch_duration_secs,\n        target_batch_duration_secs_including_fixed_cost=(\n            target_batch_duration_secs_including_fixed_cost),\n        variance=variance,\n        clock=clock,\n        record_metrics=record_metrics)\n    self._element_size_fn = element_size_fn\n    self._max_batch_dur = max_batch_duration_secs","sourceCodeStart":1069,"sourceCodeEnd":1105,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/transforms/util.py#L1069-L1105","documentation":"Raised when BatchElements is given bucket_boundaries (fixed size buckets) without a length_fn. Bucket-based size estimation needs to know how to measure each element's size; without length_fn the estimator cannot assign elements to buckets, so construction fails.","triggerScenarios":"BatchElements(bucket_boundaries=[10, 100, 1000]) without length_fn.","commonSituations":"Copying a bucket example and dropping the length_fn; assuming elements are counted by default when a custom notion of size (bytes, records, weight) is required.","solutions":["Provide length_fn, e.g. BatchElements(bucket_boundaries=[10, 100], length_fn=lambda x: len(x)).","If per-element size is not meaningful for your data, drop bucket_boundaries and use target_batch_overhead/target_batch_duration_secs instead.","Centralize the pairing of bucket_boundaries and length_fn in a helper so they are always passed together."],"exampleFix":"// before\nbeam.BatchElements(bucket_boundaries=[10, 100, 1000])\n\n// after\nbeam.BatchElements(bucket_boundaries=[10, 100, 1000],\n                   length_fn=lambda batch: len(batch))","handlingStrategy":"validation","validationCode":"if bucket_boundaries is not None and length_fn is None:\n    raise ValueError('bucket_boundaries requires length_fn')","typeGuard":"def valid_bucket_config(boundaries, length_fn):\n    return boundaries is None or (callable(length_fn) and boundaries is not None)","tryCatchPattern":"try:\n    t = beam.BatchElements(bucket_boundaries=bounds)\nexcept ValueError as e:\n    if 'length_fn' in str(e):\n        t = beam.BatchElements(bucket_boundaries=bounds, length_fn=lambda x: len(x))\n    else:\n        raise","preventionTips":["Always pass bucket_boundaries and length_fn together via a shared helper/factory.","Use buckets only when per-element size (bytes/records) is well defined; otherwise use target_* parameters.","Add a construction-time unit test for every BatchElements configuration."],"tags":["python","apache-beam","batchelements","missing-argument"],"backgroundTag":"missing-required-argument","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}