RyanCodrai/turbovec · error · ValueError

side-car value at {_crumb_path(entry)} is {obj!r}, which JSO

Error message

side-car value at {_crumb_path(entry)} is {obj!r}, which JSON cannot represent: it would be written as a bare {token} token that RFC 8259 forbids. Other JSON readers reject the file (serde_json, JSON.parse) or silently rewrite the value to null (jq). Replace it with None or a finite number before saving.

What it means

_check_json_faithful also guards values: JSON (RFC 8259) has no representation for NaN or Infinity, so json.dump would emit bare NaN/Infinity tokens that other parsers reject or rewrite to null. To guarantee the side-car is faithfully reloadable, the check raises ValueError naming the value and its path before the file is written.

Source

Thrown at turbovec-python/python/turbovec/_persist.py:329

                        f"{_crumb_path(entry)} is {type(key).__name__}, not "
                        f"str. JSON object keys are strings, so writing it "
                        f"would stringify the key and silently merge it with "
                        f"any existing {str(key)!r} key, losing data on "
                        f"reload. Convert the key to a str before saving."
                    )
                stack.append((value, entry, key))
        elif isinstance(obj, (list, tuple)):
            if id(obj) in seen:
                continue
            seen.add(id(obj))
            for i, value in enumerate(obj):
                stack.append((value, entry, i))
        elif isinstance(obj, float) and not math.isfinite(obj):
            if math.isnan(obj):
                token = "NaN"
            else:
                token = "Infinity" if obj > 0 else "-Infinity"
            raise ValueError(
                f"side-car value at {_crumb_path(entry)} is {obj!r}, "
                f"which JSON cannot represent: it would be written as a bare "
                f"{token} token that RFC 8259 forbids. Other JSON readers "
                f"reject the file (serde_json, JSON.parse) or silently "
                f"rewrite the value to null (jq). Replace it with None or a "
                f"finite number before saving."
            )


def atomic_save(index, index_path, payload: Any, sidecar_path) -> None:
    """Atomically persist an index + JSON side-car pair — the shared
    write path for all four integrations' save methods.

    The failure-ordering guarantees:

    1. ``payload`` is validated and JSON-serialized fully in memory
       *first*, so a value whose JSON form would lose data or not be
       portable JSON raises before any file is touched: a non-serializable

View on GitHub (pinned to ccab9f325e)

Solutions

Sanitize metadata before saving: replace non-finite floats with None or a finite clamped value (e.g. math.isfinite check).
Fix the source computation producing NaN/inf (guard division by zero, masked arrays, np.nan_to_num on numpy data).
Store such values as strings or a sentinel object documented as non-numeric if the semantics matter.
Wrap save in try/except ValueError and clean the reported path (given by _crumb_path) before retrying.

Example fix

// before
meta = {"score": float("inf")}
atomic_save(...)
// after
import math
meta = {"score": v if math.isfinite(v := float("inf")) else None}  # or np.nan_to_num(scores)
atomic_save(...)

Defensive patterns

Strategy: validation

Validate before calling

import math
def no_nonfinite(obj):
    if isinstance(obj, float):
        assert math.isfinite(obj), f'non-finite value: {obj}'
    elif isinstance(obj, dict):
        for v in obj.values(): no_nonfinite(v)
    elif isinstance(obj, (list, tuple)):
        for v in obj: no_nonfinite(v)
no_nonfinite(metadata)

Type guard

def json_safe_floats(obj) -> bool:
    import math
    if isinstance(obj, float):
        return math.isfinite(obj)
    if isinstance(obj, dict):
        return all(json_safe_floats(v) for v in obj.values())
    if isinstance(obj, (list, tuple)):
        return all(json_safe_floats(v) for v in obj)
    return True

Try / catch

try:
    turbovec.atomic_save(...)
except ValueError as e:
    if 'NaN' in str(e) or 'Infinity' in str(e):
        metadata = sanitize_nonfinite(metadata)  # np.nan_to_num / None
    else:
        raise

Prevention

Run np.nan_to_num or an isfinite clamp on scores before attaching them to metadata.
Avoid float('inf')/-inf as sentinels in persisted metadata; use None.
Validate metadata with a json round-trip (parse_non_standard JSON off) in tests.
Guard arithmetic that can produce NaN (0/0, inf-inf) at the source.

When it happens

Trigger: Saving (atomic_save/load/from_persist_path) a store whose metadata side-car contains float('nan'), float('inf'), or float('-inf'), typically from a distance/score computation or missing-value sentinel; the check recurses through nested metadata.

Common situations: Storing raw similarity scores that came out NaN/inf; using float('inf') as a default or sentinel in metadata; numpy float32/float64 values converted to Python floats with NaN from division by zero.

Understand the failure class

Background: "JSON serialization failed", "not JSON serializable", "Failed to serialize": why JSON marshaling errors happen and how to fix them — this error's family across 46 libraries.

Related errors

AI-assisted analysis of RyanCodrai/turbovec@ccab9f325e (2026-09-06). Data as JSON: /api/errors/90948833daf86387. Report an issue: GitHub.