pathwaycom/pathway · error · TypeError

Unsupported conversion from pw.Json to {typehints[name]}

Error message

Unsupported conversion from pw.Json to {typehints[name]}

What it means

The JSON-to-schema conversion helper in pw.utils.col builds column expressions that extract typed values from a pw.Json column according to a target schema. Only a fixed set of target dtypes has an extraction recipe (e.g. strings, ints, floats, bools, DATE_TIME_UTC via strptime, DURATION via int nanoseconds); the final `case _` raises TypeError naming the unsupported target type.

Source

Thrown at python/pathway/stdlib/utils/col.py:176

                result = _optional(
                    col,
                    lambda col: pw.unwrap(col.as_str()).dt.strptime(
                        "%Y-%m-%dT%H:%M:%S.%f"
                    ),
                )
            case dt.DATE_TIME_UTC:
                result = _optional(
                    col,
                    lambda col: pw.unwrap(col.as_str()).dt.strptime(
                        "%Y-%m-%dT%H:%M:%S.%f%z"
                    ),
                )
            case dt.DURATION:
                result = _optional(
                    col, lambda col: pw.unwrap(col.as_int()).dt.to_duration("ns")
                )
            case _:
                raise TypeError(
                    f"Unsupported conversion from pw.Json to {typehints[name]}"
                )

        return result if is_optional else pw.unwrap(result)

    colrefs = [pw.this[column_name] for column_name in schema.column_names()]
    kw = {
        colref.name: _convert_from_json(colref.name, column.get(colref.name))
        for colref in colrefs
    }
    result = column.table.select(**kw).update_types(**schema)
    return result


# TODO: generalize to apply on groupby: https://github.com/navalgo/IoT-Pathway/issues/1919
@check_arg_types
@trace_user_frame
def multiapply_all_rows(

View on GitHub (pinned to fa2f74a464)

Solutions

1. Restrict the schema passed to the conversion to the supported dtypes (str/int/float/bool/datetime/duration) and unpack exotic fields manually
2. For unsupported types, extract via column.get('field') and convert with .apply / .dt helpers instead of schema-driven conversion
3. Store genuinely non-primitive fields as pw.Json or str in the target schema and decode downstream

Example fix

# before
class S(pw.Schema):
    id: bytes  # unsupported target
out = convert(t.raw_json, schema=S)

# after
class S(pw.Schema):
    id: str
out = convert(t.raw_json, schema=S)

from pathway import dt
SUPPORTED = {dt.STRING, dt.INT, dt.FLOAT, dt.BOOL, dt.DATE_TIME_UTC, dt.DATE_TIME_NAIVE, dt.DURATION}
bad = [n for n, t in schema.typehints().items() if dt.eval_type(t) not in SUPPORTED]
assert not bad, f'unsupported target dtypes for json conversion: {bad}'

def schema_is_json_convertible(schema: type) -> bool:
    from pathway import dt
    SUPPORTED = {dt.STRING, dt.INT, dt.FLOAT, dt.BOOL, dt.DATE_TIME_UTC, dt.DATE_TIME_NAIVE, dt.DURATION}
    return all(dt.eval_type(t) in SUPPORTED for t in schema.typehints().values())

try:
    result = convert(json_col, schema=S)
except TypeError as e:
    if 'Unsupported conversion from pw.Json' in str(e):
        raise TypeError(f'schema {S} has a non-JSON-representable column') from e
    raise

When it happens

Trigger: Calling the json-unpacking API (pw.utils.col based, e.g. unpack/parse with schema=) where the schema declares bytes, a Pointer, an array/map dtype, or another exotic type; targeting DATE_TIME_NAIVE patterns that do not match the supported strptime format when no case exists for that dtype.

Common situations: Feeding ORM/dataclass schemas containing bytes or UUID-like fields into a JSON unpacker; schema-first design where the schema was generated from a DB model rather than restricted to JSON-representable types; API responses with nested objects mapped to columns not covered by the converter.

Related errors

• Cannot convert Json {self.value} to {type}
• Object in {expression!r} has to be a JSON or sequence.
• 'json_field_paths' references field {field_name!r} which is
• Failed to detect the region of S3 bucket {bucket!r} (HTTP st
• SchemaRegistryHeader.value must be a str, got {type(self.val

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/b81bce02b3476567. Report an issue: GitHub.