run-llama/llama_index · error · ValueError

Malformed partial JSON encountered.

Error message

Malformed partial JSON encountered.

What it means

Inside replace_errors/json repair logic used for partial (in-flight) JSON — as streamed structured output arrives, llama-index tracks brackets/quotes and appends missing closers. If a closing '}' or ']' appears that does not match the top of the expected-closer stack (e.g. ']' when a '}' is pending, ignoring matching pairs inside strings), the input cannot be repaired deterministically and ValueError is raised.

Source

Thrown at llama-index-core/llama_index/core/llms/utils.py:170

                char = "\\n"  # Replace the newline character with the escape sequence.
            elif char == "\\":
                escaped = not escaped
            else:
                escaped = False
        else:
            if char == '"':
                is_inside_string = True
                escaped = False
            elif char == "{":
                stack.append("}")
            elif char == "[":
                stack.append("]")
            elif char == "}" or char == "]":
                if stack and stack[-1] == char:
                    stack.pop()
                else:
                    # Mismatched closing character; the input is malformed.
                    raise ValueError("Malformed partial JSON encountered.")

        # Append the processed character to the new string.
        new_s += char

    # If we're still inside a string at the end of processing and no colon was found after the opening quote,
    # this is an incomplete key - remove it
    if is_inside_string and '"' in new_s and ":" not in new_s[new_s.rindex('"') :]:
        new_s = new_s[: new_s.rindex('"')]
    elif is_inside_string:
        new_s += '"'

    # Check if we have an incomplete key-value pair
    new_s = new_s.rstrip()
    if new_s.endswith(":"):
        new_s += " null"  # Add a default value for incomplete value
    elif new_s.endswith(","):
        new_s = new_s[:-1]  # Remove the trailing comma

View on GitHub (pinned to afd0fef371)

Solutions

  1. Feed the parser strictly the prefix accumulated so far, in order — never skip or reorder chunks.
  2. Wait for the stream to finish and json.loads the complete text instead of repairing partials.
  3. Catch ValueError per-chunk and skip intermediate unparseable prefixes; retry only on the final buffer.
  4. Use native structured output (function calling / JSON mode) so repair is unnecessary.

Example fix

# before
partial = chunks[0] + chunks[2]  # dropped chunk -> mismatched brackets
obj = repair_json(partial)
# after
partial = "".join(chunks[:i])  # always an in-order prefix
try:
    obj = repair_json(partial)
except ValueError:
    continue  # skip unparseable intermediate prefix
Defensive patterns

Strategy: try-catch

Validate before calling

def is_ordered_json_prefix(buffer: str, new_chunk: str) -> bool:
    # crude guard: only append chunks that continue the current buffer position
    return isinstance(new_chunk, str) and (buffer == "" or True)  # order is caller's job; validate brackets below

def brackets_balanced_or_open(s: str) -> bool:
    stack = []
    in_str = False
    esc = False
    for ch in s:
        if esc:
            esc = False
        elif in_str and ch == "\\":
            esc = True
        elif ch == '"':
            in_str = not in_str
        elif not in_str and ch in "{[":
            stack.append(ch)
        elif not in_str and ch in "}":
            if not stack or stack.pop() != "{":
                return False
        elif not in_str and ch == "]":
            if not stack or stack.pop() != "[":
                return False
    return True

Type guard

def looks_like_json_stream_prefix(text: str) -> bool:
    return text.lstrip().startswith(("{", "["))

Try / catch

try:
    obj = repair_partial_json(buffer)
except ValueError:
    obj = None  # intermediate prefix unrepairable; wait for more tokens
if obj is None and stream_finished:
    raise  # final buffer must parse or the output is genuinely malformed

Prevention

When it happens

Trigger: Streaming a JSON object where the model emits mismatched brackets mid-stream, or feeding partially accumulated stream text that was cut/concatenated incorrectly (dropped or duplicated chunks) into this repair path.

Common situations: Accumulating streamed structured output manually and passing truncated/mangled buffers; models that emit invalid JSON skeletons; chunk handling that reorders or drops tokens.

Understand the failure class

Related errors

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/686720ac7db08515. Report an issue: GitHub.