apache/beam · error · ValueError

Non-path arguments must be passed by keyword for splittable…

Error message

Non-path arguments must be passed by keyword for splittable csv reads.

What it means

The splittable CSV splitter only supports keyword-formatted options; if positional args (the pandas *args path) are passed to a splittable read_csv, it raises ValueError because positional options cannot be mapped to per-chunk parsing.

Solutions

Convert every pandas option to keyword form: read_csv(path, sep=',', names=[...], splittable=True).
Drop splittable=True if you must keep positional args (loses splitting).
Audit wrapper functions that forward *args into read_csv when splittable is enabled.

Example fix

// before
read_csv('f.csv', ',', splittable=True)
// after
read_csv('f.csv', sep=',', splittable=True)

Defensive patterns

Strategy: validation

Validate before calling

if splittable and args:
    raise ValueError('Pass csv options as keywords when splittable=True')

Try / catch

try:
    df = read_csv(path, *args, **kwargs)
except ValueError as e:
    if 'by keyword' in str(e):
        kwargs.update(dict(zip(('sep',), args)))
        df = read_csv(path, splittable=True, **kwargs)
    else:
        raise

Prevention

Always use keyword arguments for csv options
Avoid wrappers forwarding *args to splittable reads
Set splittable=True only for large, newline-safe files

When it happens

Trigger: Calling read_csv(path, sep=',', splittable=True) — sep passed positionally as *args rather than as a keyword — or forwarding a tuple of pandas args along with splittable=True.

Common situations: Code converted from pandas positional style; wrappers that pass *args through to read_csv while enabling splittable=True for large files.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/2598b3f76ba009b0. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/dataframe/io.py:405


def _maybe_encode(str_or_bytes):
  if isinstance(str_or_bytes, str):
    return str_or_bytes.encode('utf-8')
  else:
    return str_or_bytes


class _TextFileSplitter(_DelimSplitter):
  """Splitter for dynamically sharding CSV files and newline record boundaries.

  Currently does not handle quoted newlines, so is off by default, but such
  support could be added in the future.
  """
  def __init__(self, args, kwargs, read_chunk_size=_DEFAULT_BYTES_CHUNKSIZE):
    if args:
      # TODO(robertwb): Automatically populate kwargs as we do for df methods.
      raise ValueError(
          'Non-path arguments must be passed by keyword '
          'for splittable csv reads.')
    if kwargs.get('skipfooter', 0):
      raise ValueError('Splittablility incompatible with skipping footers.')
    super().__init__(
        _maybe_encode(kwargs.get('lineterminator', b'\n')),
        _DEFAULT_BYTES_CHUNKSIZE)
    self._kwargs = kwargs

  def read_header(self, handle):
    if self._kwargs.get('header', 'infer') == 'infer':
      if 'names' in self._kwargs:
        header = None
      else:
        header = 0
    else:
      header = self._kwargs['header']

View on GitHub (pinned to 12126d8942)