{"record":{"id":"ee593bce3c81f03b","repo":"pathwaycom/pathway","slug":"json-field-paths-references-field-field-name-r","errorCode":null,"errorMessage":"'json_field_paths' references field {field_name!r} which is not in the schema. Known fields: {sorted(schema_columns)}.","messagePattern":"'json_field_paths' references field (.+?) which is not in the schema\\. Known fields: (.+?)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/pathway/io/_utils.py","lineNumber":387,"sourceCode":"            raise ValueError(\"Unexpected argument for json format: csv_settings\")\n        if autogenerate_key:\n            raise ValueError(\n                f\"'autogenerate_key' is only meaningful for 'raw' or \"\n                f\"'plaintext' formats and would have no effect with \"\n                f\"{format!r}. Drop it or pick a compatible format.\"\n            )\n        if json_field_paths is not None:\n            schema_columns = set(schema.column_names())\n            for field_name, path in json_field_paths.items():\n                if field_name == METADATA_COLUMN_NAME:\n                    raise ValueError(\n                        f\"'json_field_paths' cannot be used for \"\n                        f\"{METADATA_COLUMN_NAME!r}: the connector populates \"\n                        f\"this column itself when 'with_metadata=True', so \"\n                        f\"any JSON path would be silently ignored.\"\n                    )\n                if field_name not in schema_columns:\n                    raise ValueError(\n                        f\"'json_field_paths' references field {field_name!r} \"\n                        f\"which is not in the schema. Known fields: \"\n                        f\"{sorted(schema_columns)}.\"\n                    )\n                if path != \"\" and not path.startswith(\"/\"):\n                    raise ValueError(\n                        f\"Invalid JSON Pointer for field {field_name!r}: \"\n                        f\"{path!r}. JSON Pointers (RFC 6901) must be empty \"\n                        f\"or start with '/' (e.g. '/foo/bar').\"\n                    )\n        return schema, api.DataFormat(\n            **api_schema,\n            format_type=data_format_type,\n            column_paths=json_field_paths,\n            schema_registry_settings=maybe_schema_registry_settings(\n                schema_registry_settings\n            ),\n        )","sourceCodeStart":369,"sourceCodeEnd":405,"githubUrl":"https://github.com/pathwaycom/pathway/blob/fa2f74a4649b7c5908690cf60137263d8d80de5f/python/pathway/io/_utils.py#L369-L405","documentation":"Raised by Pathway's input-connector helpers when the 'json_field_paths' dict maps a field name that is not a column of the parsed schema. 'json_field_paths' tells the connector where each output column's value lives inside the incoming JSON document, so every key of that dict must match a column name produced by the schema (the connector's schema or the one you supplied). The error lists all known field names to make the mismatch obvious.","triggerScenarios":"Calling an input connector (e.g. pw.io.kafka.read, pw.io.http.read) with both a schema and json_field_paths={'payload_user': '/user'} where 'payload_user' is not a column of that schema; or renaming/removing a schema column without updating json_field_paths; or misspelling a field name in the mapping.","commonSituations":"Copy-pasting a json_field_paths dict from an example whose schema differs from yours; evolving the schema (column renamed) while leaving stale entries in json_field_paths; assuming json_field_paths creates new columns instead of only binding existing ones.","solutions":["Compare the keys of json_field_paths against the sorted list of known fields printed in the error message and fix or remove the offending entry.","If you intended the field to exist, add it to the schema passed to the connector (or adjust the connector's schema definition) so the name matches.","If you meant to extract a nested value, keep the field name identical to the schema column and only change the JSON Pointer path value."],"exampleFix":"# before\nschema = tb.schema_from_types(user=str, amount=int)\nt = pw.io.kafka.read(..., schema=schema,\n    json_field_paths={'username': '/user'})  # 'username' not in schema\n\n# after\nschema = tb.schema_from_types(username=str, amount=int)\nt = pw.io.kafka.read(..., schema=schema,\n    json_field_paths={'username': '/user'})  # key matches schema column","handlingStrategy":"validation","validationCode":"def validate_json_field_paths(schema, json_field_paths):\n    known = set(schema.column_names())\n    unknown = set(json_field_paths) - known\n    assert not unknown, f\"json_field_paths fields not in schema: {sorted(unknown)}; known: {sorted(known)}\"","typeGuard":"def paths_match_schema(schema, json_field_paths: dict) -> bool:\n    return set(json_field_paths).issubset(set(schema.column_names()))","tryCatchPattern":"try:\n    t = pw.io.kafka.read(..., schema=schema, json_field_paths=paths)\nexcept ValueError as e:\n    if \"not in the schema\" in str(e):\n        raise SystemExit(f\"Fix json_field_paths keys: {e}\")\n    raise","preventionTips":["Derive json_field_paths keys from schema.column_names() instead of writing them by hand.","After any schema change, re-run a dry validation that every json_field_paths key is still a schema column."],"tags":["pathway","connector","json","schema","validation"],"backgroundTag":null,"analyzedSha":"fa2f74a4649b7c5908690cf60137263d8d80de5f","analyzedAt":"2026-08-15T01:48:17.006Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}