apache/beam · error · TypeError

"Packages must be an iterable of strings, got %r" % packages

Error message

"Packages must be an iterable of strings, got %r" % packages

What it means

Raised by the PythonPackageProvider (venv-based external provider) constructor when the 'packages' argument is not an iterable or is itself a plain string. A string is explicitly rejected because iterating it would treat each character as a package name.

Solutions

1. Wrap the value in a list: PythonPackageProvider(['pandas==1.0']).
2. In YAML provider config, write packages as a list: packages: ['pandas', 'numpy'].
3. If the input may be a scalar, convert defensively: packages = [packages] if isinstance(packages, str) else list(packages).

Example fix

// before
PythonPackageProvider('pandas==2.0')
// after
PythonPackageProvider(['pandas==2.0'])

def normalize_packages(packages):
    if isinstance(packages, str):
        packages = [packages]
    if not isinstance(packages, Iterable):
        raise TypeError('packages must be an iterable of strings')
    return list(packages)

def is_valid_packages_arg(packages):
    return isinstance(packages, Iterable) and not isinstance(packages, str)

try:
    provider = PythonPackageProvider(packages)
except TypeError as e:
    if 'Packages must be an iterable' in str(e):
        logging.warning('Wrapping scalar packages into a list')
        provider = PythonPackageProvider([packages])
    else:
        raise

When it happens

Trigger: Constructing PythonPackageProvider with packages=None, an int, or a single string like 'pandas==1.0' instead of a list of package specs; or passing a comma-joined string from config instead of a YAML list.

Common situations: YAML provider specs where packages was written as a scalar instead of a list, or programmatic construction where a single package was passed without wrapping in a list.

Related errors

• 'Transform mapping must be a dict.'
• A sink must inherit iobase.Sink, iobase.NativeSink, or be a…
• Cannot encode payload for WriteToPubSub. Expected valid…
• Cannot interpret as seconds.
• Cannot interpret as subseconds.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/1003311d9206e773. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/yaml_provider.py:1345

class PypiExpansionService:
  """Expands transforms by fully qualified name in a virtual environment
  with the given dependencies.
  """
  if 'TOX_WORK_DIR' in os.environ:
    VENV_CACHE = tempfile.mkdtemp(
        prefix='test-venv-cache-', dir=os.environ['TOX_WORK_DIR'])
  elif 'RUNNER_WORKDIR' in os.environ:
    VENV_CACHE = tempfile.mkdtemp(
        prefix='test-venv-cache-', dir=os.environ['RUNNER_WORKDIR'])
  else:
    VENV_CACHE = os.path.expanduser("~/.apache_beam/cache/venvs")

  def __init__(
      self, packages: Iterable[str], base_python: str = sys.executable):
    if not isinstance(packages, Iterable) or isinstance(packages, str):
      raise TypeError(
          "Packages must be an iterable of strings, got %r" % packages)
    self._packages = list(packages)
    self._base_python = base_python

  @classmethod
  def _key(cls, base_python: str, packages: list[str]) -> str:
    def normalize_package(package):
      if os.path.exists(package):
        # Ignore the exact path by which this package was referenced,
        # but do create a new environment if it changed.
        with open(package, 'rb') as fin:
          return os.path.basename(package) + '-' + _file_digest(
              fin, 'sha256').hexdigest()
      else:
        # Assume urls and pypi identifiers are immutable.
        return package

    return json.dumps({

View on GitHub (pinned to 12126d8942)