pathwaycom/pathway · error · ValueError

Failed to parse DType from dict: {data}

Error message

Failed to parse DType from dict: {data}

What it means

parse_dtype_from_dict reconstructs a DType from its serialized dictionary form (a {'type': <NAME>} payload used when persisting/communicating schemas). It only knows the fixed sets NON_COMPOSITE_DTYPES and COMPOSITE_DTYPES; an unknown 'type' name means the payload was produced by a different (usually newer or older) Pathway version or was hand-crafted/edited.

Source

Thrown at python/pathway/internals/dtype.py:1101

    "BYTES": BYTES,
    "DATE_TIME_NAIVE": DATE_TIME_NAIVE,
    "DATE_TIME_UTC": DATE_TIME_UTC,
    "DURATION": DURATION,
    "Json": JSON,
    "NONE": NONE,
    "ANY": ANY,
    "PY_OBJECT_WRAPPER": ANY_PY_OBJECT_WRAPPER,
    "POINTER": ANY_POINTER,
}


def parse_dtype_from_dict(data: dict) -> DType:
    type_name = data["type"]
    if type_name in NON_COMPOSITE_DTYPES:
        return NON_COMPOSITE_DTYPES[type_name]
    if type_name in COMPOSITE_DTYPES:
        return COMPOSITE_DTYPES[type_name](data)
    raise ValueError(f"Failed to parse DType from dict: {data}")

View on GitHub (pinned to fa2f74a464)

Solutions

1. Pin all processes that read/write the serialized dtypes to the same Pathway version (check pip show pathway everywhere)
2. Regenerate the serialized schema/dict from scratch in the current version instead of reusing old files
3. If the payload must be read, pre-validate data['type'] against the keys in pathway.internals.dtype.NON_COMPOSITE_DTYPES / COMPOSITE_DTYPES and migrate unknown names explicitly

Example fix

// before
dtype = parse_dtype_from_dict(loaded_payload)
// after
from pathway.internals.dtype import NON_COMPOSITE_DTYPES, COMPOSITE_DTYPES, parse_dtype_from_dict
assert loaded_payload["type"] in {**NON_COMPOSITE_DTYPES, **COMPOSITE_DTYPES}, f"dtype from another version: {loaded_payload['type']}"
dtype = parse_dtype_from_dict(loaded_payload)

from pathway.internals.dtype import NON_COMPOSITE_DTYPES, COMPOSITE_DTYPES

def can_parse_dtype(data: dict) -> bool:
    return isinstance(data, dict) and data.get("type") in {**NON_COMPOSITE_DTYPES, **COMPOSITE_DTYPES}

try:
    parse_dtype_from_dict(data)
except ValueError as e:
    if "Failed to parse DType" in str(e):
        log.error("serialized dtype from a different pathway version", payload=data)

When it happens

Trigger: Restoring a persisted state or replaying a schema dump created by a different Pathway version that introduced a dtype name this build's dicts don't contain; manually constructing or mutating the serialized dtype dict; corrupted IPC/serialization payloads crossing version boundaries.

Common situations: Upgrading/downgrading Pathway while keeping persistent storage; running mixed-version clusters or worker containers; editing serialized pipeline definitions by hand.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Column {api.DIFF_PSEUDOCOLUMN} can only have 1 and -1 values
• demo.replay_csv_with_time: unit should be either 's', 'ms, '
• Cannot flatten column of type {dtype}.
• unexpected value of PATHWAY_SNAPSHOT_ACCESS environment vari
• Unsupported type {input_type}, use pw.DATE_TIME_UTC or pw.DA

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/746102c6dfe6c6ae. Report an issue: GitHub.