apache/beam · error · ValueError

Error parsing string

Error message

Error parsing string:
{raw_str}
{e}

What it means

parse_chunk_strings evaluates each cleaned string with eval() inside safe_globals and expects the result to be a langchain Chunk instance. Any exception during parsing/eval, or a parsed object that is not a Chunk, is re-raised as this ValueError with the original raw string and the underlying error.

Solutions

1. Inspect the raw_str in the message and fix the stored string so it evaluates to a Chunk instance.
2. Re-serialize the chunks with the same Chunk class/version used to write them, then rerun the pipeline.
3. If the source data is plain dicts, wrap them in Chunk(...) before passing strings to parse_chunk_strings.
4. Catch ValueError per string and skip/log malformed entries instead of failing the whole batch.

Example fix

// before
chunks = parse_chunk_strings(['{"content": "text"}'])
// after
from apache_beam.ml.rag.chunking import Chunk
raw = '{"content": "text"}'
import json
chunks = [Chunk(content=d['content']) for d in [json.loads(raw)]]

import ast
for s in strings:
    try:
        ast.parse(s)
    except SyntaxError as e:
        raise ValueError(f'Unparseable chunk string: {s[:80]}...') from e

def is_chunk_str(s: str) -> bool:
    import ast
    try:
        ast.parse(s)
        return True
    except SyntaxError:
        return False

try:
    chunks = parse_chunk_strings(raw_strings)
except ValueError as e:
    logger.error('Chunk parse failed: %s', e)
    chunks = []

When it happens

Trigger: Calling parse_chunk_strings with strings that are not valid Python literals/expressions, strings that evaluate to something other than a Chunk (e.g. a plain dict or str), or strings referencing names not present in safe_globals.

Common situations: Embedding data serialized by a different pipeline version that stored dicts instead of Chunk objects; corrupted or truncated chunk strings in a Milvus-backed RAG index; hand-edited chunk records.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• Parsed object is not a Chunk instance
• Approximate Nearest Neighbor Search (ANNS) field must be…
• chunk_to_dict_fn is deprecated, use embeddable_to_dict_fn
• Collection name must be provided
• Collection name must be provided

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0bb13479c9c0769b. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/ml/rag/utils.py:117

      'defaultdict': defaultdict,
      'list': list,
      '__builtins__': {}
  }

  for raw_str in chunk_str_list:
    try:
      # replace "<class 'list'>" with actual list reference.
      cleaned_str = re.sub(
          r"defaultdict\(<class 'list'>", "defaultdict(list", raw_str)

      # Evaluate string in restricted environment.
      chunk = eval(cleaned_str, safe_globals)  # pylint: disable=eval-used
      if isinstance(chunk, Chunk):
        parsed_chunks.append(chunk)
      else:
        raise ValueError("Parsed object is not a Chunk instance")
    except Exception as e:
      raise ValueError(f"Error parsing string:\n{raw_str}\n{e}")

  return parsed_chunks


def unpack_dataclass_with_kwargs(dataclass_instance):
  """Unpacks dataclass fields into a flat dict, merging kwargs with precedence.

  Args:
    dataclass_instance: Dataclass instance to unpack.

  Returns:
    dict: Flattened dictionary with kwargs taking precedence over fields.
  """
  # Create a copy of the dataclass's __dict__.
  params_dict: dict = dataclass_instance.__dict__.copy()

  # Extract the nested kwargs dictionary.
  nested_kwargs = params_dict.pop('kwargs', {})

View on GitHub (pinned to 12126d8942)