pathwaycom/pathway · error · ValueError

'json_field_paths' cannot be used for {METADATA_COLUMN_NAME!

Error message

'json_field_paths' cannot be used for {METADATA_COLUMN_NAME!r}: the connector populates this column itself when 'with_metadata=True', so any JSON path would be silently ignored.

What it means

Error "'json_field_paths' cannot be used for {METADATA_COLUMN_NAME!r}: the connector populates this column itself when 'with_metadata=True', so any JSON path would be silently ignored." thrown in pathwaycom/pathway.

Source

Thrown at python/pathway/io/_utils.py:380

            delimiter=",",
            schema_registry_settings=maybe_schema_registry_settings(
                schema_registry_settings
            ),
        )
    elif data_format_type == "jsonlines":
        if csv_settings is not None:
            raise ValueError("Unexpected argument for json format: csv_settings")
        if autogenerate_key:
            raise ValueError(
                f"'autogenerate_key' is only meaningful for 'raw' or "
                f"'plaintext' formats and would have no effect with "
                f"{format!r}. Drop it or pick a compatible format."
            )
        if json_field_paths is not None:
            schema_columns = set(schema.column_names())
            for field_name, path in json_field_paths.items():
                if field_name == METADATA_COLUMN_NAME:
                    raise ValueError(
                        f"'json_field_paths' cannot be used for "
                        f"{METADATA_COLUMN_NAME!r}: the connector populates "
                        f"this column itself when 'with_metadata=True', so "
                        f"any JSON path would be silently ignored."
                    )
                if field_name not in schema_columns:
                    raise ValueError(
                        f"'json_field_paths' references field {field_name!r} "
                        f"which is not in the schema. Known fields: "
                        f"{sorted(schema_columns)}."
                    )
                if path != "" and not path.startswith("/"):
                    raise ValueError(
                        f"Invalid JSON Pointer for field {field_name!r}: "
                        f"{path!r}. JSON Pointers (RFC 6901) must be empty "
                        f"or start with '/' (e.g. '/foo/bar')."
                    )
        return schema, api.DataFormat(

View on GitHub (pinned to fa2f74a464)

Solutions

1. Remove the '_metadata' entry from json_field_paths; the connector fills that column itself when with_metadata=True.

Example fix

pw.io.fs.read(path, format='json', schema=MySchema, with_metadata=True, json_field_paths={'data': '/data'})

When it happens

Trigger: Thrown at python/pathway/io/_utils.py:380 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/90bec10e31c9600c. Report an issue: GitHub.