apache/beam · error · ValueError

escapechar must be bytes of size 1: '%s'

Error message

escapechar must be bytes of size 1: '%s'

What it means

The text/batch reader validates its CSV escapechar option: Python's csv module requires escapechar to be a one-byte string, so anything longer (or of another type) is rejected at constructor time as unusable for delimiter escaping.

Source

Thrown at sdks/python/apache_beam/io/textio.py:170

    self._buffer_size = buffer_size
    if skip_header_lines < 0:
      raise ValueError(
          'Cannot skip negative number of header lines: %d' % skip_header_lines)
    elif skip_header_lines > 10:
      _LOGGER.warning(
          'Skipping %d header lines. Skipping large number of header '
          'lines might significantly slow down processing.')
    self._skip_header_lines = skip_header_lines
    self._header_matcher, self._header_processor = header_processor_fns
    if delimiter is not None:
      if not isinstance(delimiter, bytes) or len(delimiter) == 0:
        raise ValueError('Delimiter must be a non-empty bytes sequence.')
      if self._is_self_overlapping(delimiter):
        raise ValueError('Delimiter must not self-overlap.')
    self._delimiter = delimiter
    if escapechar is not None:
      if not (isinstance(escapechar, bytes) and len(escapechar) == 1):
        raise ValueError(
            "escapechar must be bytes of size 1: '%s'" % escapechar)
    self._escapechar = escapechar

  def display_data(self):
    parent_dd = super().display_data()
    parent_dd['strip_newline'] = DisplayDataItem(
        self._strip_trailing_newlines, label='Strip Trailing New Lines')
    parent_dd['buffer_size'] = DisplayDataItem(
        self._buffer_size, label='Buffer Size')
    parent_dd['coder'] = DisplayDataItem(self._coder.__class__, label='Coder')
    return parent_dd

  def read_records(self, file_name, range_tracker):
    start_offset = range_tracker.start_position()
    read_buffer = _TextSource.ReadBuffer(b'', 0)

    next_record_start_position = -1

View on GitHub (pinned to 12126d8942)

Solutions

1. Pass a single-byte bytes value, e.g. escapechar=b'"' or escapechar=b'\\'
2. Encode a str char: escapechar=c.encode('ascii') ensuring len == 1
3. Pass escapechar=None if no escaping is needed

Example fix

// before
ReadFromText('gs://bucket/f', escapechar='\\')
// after
ReadFromText('gs://bucket/f', escapechar=b'\\')

if escapechar is not None and (not isinstance(escapechar, bytes) or len(escapechar) != 1):
    raise ValueError('escapechar must be a single byte, got %r' % (escapechar,))

def is_valid_escapechar(c) -> bool:
    return c is None or (isinstance(c, bytes) and len(c) == 1)

try:
    src = ReadFromText(path, escapechar=esc)
except ValueError as e:
    if 'escapechar' in str(e):
        esc = esc.encode('ascii') if isinstance(esc, str) and len(esc) == 1 else None
        src = ReadFromText(path, escapechar=esc)
    else:
        raise

When it happens

Trigger: Passing escapechar='\\' (str) instead of b'\\', or a multi-byte bytes value like b'\\\\', or None-like sequences of wrong length.

Common situations: Python 3 str-vs-bytes confusion when configuring quote/escape behavior for CSV-style sources.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Delimiter must be a non-empty bytes sequence.
• Delimiter must not self-overlap.
• MatchContinuously interval must be positive.
• Please install apache_beam[dataframe]
• The --setup_file option expects the full path to a file name

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f987e5c3a2a33a8f. Report an issue: GitHub.