apache/beam · error · TypeError

start_offset must be a number. Received: %r

Error message

start_offset must be a number. Received: %r

What it means

_FileBasedSource.__init__ requires start_offset to be an int and raises TypeError when it is not. Offsets are byte/record positions used for dynamic work rebalancing, so a non-numeric start offset is rejected immediately. Note that bools are ints in Python but floats/strings/None are not accepted.

Solutions

  1. Pass an integer: convert with int(start_offset) or use floor division //
  2. Validate the offset type before constructing the source
  3. Fix the upstream config/parser so offsets are parsed as ints

Example fix

// before
src = _FileBasedSource(path, file_size / 2, file_size)
// after
src = _FileBasedSource(path, int(file_size // 2), file_size)
Defensive patterns

Strategy: validation

Validate before calling

if not isinstance(start_offset, int) or isinstance(start_offset, bool):
    raise TypeError(f'start_offset must be int, got {type(start_offset).__name__}')

Type guard

def is_valid_offset(v) -> bool:
    return isinstance(v, int) and not isinstance(v, bool) and v >= 0

Try / catch

try:
    src = _FileBasedSource(path, start, stop)
except TypeError as e:
    if 'start_offset must be a number' in str(e):
        start = int(start)
        src = _FileBasedSource(path, start, stop)
    else:
        raise

Prevention

When it happens

Trigger: Constructing a _FileBasedSource (or subclass like _TextSource/_AvroSource) with start_offset as a float, string, or None; passing a value read from config/CLI as a string; arithmetic producing a float (e.g. file_size / 2).

Common situations: Deserializing offsets from JSON where they become strings or floats; dividing file sizes with '/' instead of '//'; passing None after a failed lookup.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/03e685ad99ec6054. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/filebasedsource.py:252

def _determine_splittability_from_compression_type(file_path, compression_type):
  if compression_type == CompressionTypes.AUTO:
    compression_type = CompressionTypes.detect_compression_type(file_path)

  return compression_type == CompressionTypes.UNCOMPRESSED


class _SingleFileSource(iobase.BoundedSource):
  """Denotes a source for a specific file type."""
  def __init__(
      self,
      file_based_source,
      file_name,
      start_offset,
      stop_offset,
      min_bundle_size=0,
      splittable=True):
    if not isinstance(start_offset, int):
      raise TypeError(
          'start_offset must be a number. Received: %r' % start_offset)
    if stop_offset != range_trackers.OffsetRangeTracker.OFFSET_INFINITY:
      if not isinstance(stop_offset, int):
        raise TypeError(
            'stop_offset must be a number. Received: %r' % stop_offset)
      if start_offset >= stop_offset:
        raise ValueError(
            'start_offset must be smaller than stop_offset. Received %d and %d '
            'for start and stop offsets respectively' %
            (start_offset, stop_offset))

    self._file_name = file_name
    self._is_gcs_file = file_name.startswith('gs://') if file_name else False
    self._start_offset = start_offset
    self._stop_offset = stop_offset
    self._min_bundle_size = min_bundle_size
    self._file_based_source = file_based_source
    self._splittable = splittable

View on GitHub (pinned to 12126d8942)