What causes "Delimiter must be a non-empty bytes sequence." in apache/beam?

Pass a bytes value, e.g. delimiter=b'\n' or delimiter=b','

apache/beam · error · ValueError

Delimiter must be a non-empty bytes sequence.

Error message

Delimiter must be a non-empty bytes sequence.

What it means

In _TextSource.__init__, a custom record delimiter passed to a text-based I/O transform must be a non-empty bytes object. The library throws ValueError immediately at construction when delimiter is not bytes or is empty, because delimiters are used in binary scanning of file data.

Source

Thrown at sdks/python/apache_beam/io/textio.py:164

        compression_type=compression_type,
        validate=validate)

    self._strip_trailing_newlines = strip_trailing_newlines
    self._compression_type = compression_type
    self._coder = coder
    self._buffer_size = buffer_size
    if skip_header_lines < 0:
      raise ValueError(
          'Cannot skip negative number of header lines: %d' % skip_header_lines)
    elif skip_header_lines > 10:
      _LOGGER.warning(
          'Skipping %d header lines. Skipping large number of header '
          'lines might significantly slow down processing.')
    self._skip_header_lines = skip_header_lines
    self._header_matcher, self._header_processor = header_processor_fns
    if delimiter is not None:
      if not isinstance(delimiter, bytes) or len(delimiter) == 0:
        raise ValueError('Delimiter must be a non-empty bytes sequence.')
      if self._is_self_overlapping(delimiter):
        raise ValueError('Delimiter must not self-overlap.')
    self._delimiter = delimiter
    if escapechar is not None:
      if not (isinstance(escapechar, bytes) and len(escapechar) == 1):
        raise ValueError(
            "escapechar must be bytes of size 1: '%s'" % escapechar)
    self._escapechar = escapechar

  def display_data(self):
    parent_dd = super().display_data()
    parent_dd['strip_newline'] = DisplayDataItem(
        self._strip_trailing_newlines, label='Strip Trailing New Lines')
    parent_dd['buffer_size'] = DisplayDataItem(
        self._buffer_size, label='Buffer Size')
    parent_dd['coder'] = DisplayDataItem(self._coder.__class__, label='Coder')
    return parent_dd

View on GitHub (pinned to 12126d8942)

Solutions

Pass a bytes value, e.g. delimiter=b'\n' or delimiter=b','
Use a non-empty bytes delimiter; to disable delimiter handling pass delimiter=None
Encode your string delimiter: delimiter=my_str.encode('utf-8')

Example fix

// before
ReadFromText('gs://bucket/file', delimiter=',')
// after
ReadFromText('gs://bucket/file', delimiter=b',')

Defensive patterns

Strategy: validation

Validate before calling

if delimiter is not None and (not isinstance(delimiter, bytes) or len(delimiter) == 0):
    raise ValueError('delimiter must be a non-empty bytes object, got %r' % (delimiter,))

Type guard

def is_valid_delimiter(d) -> bool:
    return d is None or (isinstance(d, bytes) and len(d) > 0)

Try / catch

try:
    src = ReadFromText(path, delimiter=delim)
except ValueError as e:
    if 'Delimiter' in str(e):
        delim = delim.encode('utf-8') if isinstance(delim, str) else None
        src = ReadFromText(path, delimiter=delim)
    else:
        raise

Prevention

Always use bytes literals (b',', b'\t') for delimiters in Python 3
Remember delimiter=None disables custom delimiters — don't pass ''
Unit-test I/O transforms construction with the exact args used in production

When it happens

Trigger: Passing delimiter='' to ReadFromText/WriteToText, or passing a str like ',' instead of b',', or any non-bytes sequence such as a list.

Common situations: Copy-pasting examples with string delimiters in Python 3 (where str != bytes), or trying to disable delimiter handling by passing an empty string.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/2b9a6daf56c9e19e. Report an issue: GitHub.