different-ai/openwork · warning · SafeProbeFailure

invalid_utf8

invalid_utf8

Error message

invalid_utf8

What it means

After accumulating the bounded bytes, readBoundedBody decodes them with a fatal UTF-8 TextDecoder. If the bytes are not valid UTF-8 (e.g., gzip/binary payload), decoding throws and SafeProbeFailure('invalid_utf8') is raised. The probe only understands textual (JSON/SSE) responses.

Source

Thrown at apps/server/src/agent-context-cloud-probe.ts:483

    if (error instanceof SafeProbeFailure) throw error;
    // Invoke cancellation even after the deadline has fired. Do not await it:
    // a hostile stream can ignore both AbortSignal and cancellation.
    void reader.cancel().catch(() => undefined);
    if (error instanceof ProbeTimeout || deadline.timedOut()) throw new ProbeTimeout();
    // Mid-body transport failures keep their original cause so the outer
    // classifier can attribute them to the network layer, not the protocol.
    throw error;
  }
  const bytes = new Uint8Array(size);
  let offset = 0;
  for (const chunk of chunks) {
    bytes.set(chunk, offset);
    offset += chunk.byteLength;
  }
  try {
    return new TextDecoder("utf-8", { fatal: true }).decode(bytes);
  } catch {
    throw new SafeProbeFailure("invalid_utf8");
  }
}

function parseJson(text: string): unknown {
  try {
    return JSON.parse(text);
  } catch {
    throw new SafeProbeFailure("invalid_json");
  }
}

function parseSse(text: string): unknown {
  const messages: unknown[] = [];
  let event = "";
  let data: string[] = [];
  const dispatch = () => {
    if (data.length === 0) {
      event = "";

View on GitHub (pinned to 2b7df46e8a)

Solutions

1. Probe a textual JSON endpoint, not a binary/compressed one.
2. Ensure the request sets Accept headers the server honors (application/json, identity encoding).
3. Decompress manually if the transport cannot (e.g., DecompressionStream) before decoding.

Example fix

// before
try {
  return new TextDecoder("utf-8", { fatal: true }).decode(bytes);
} catch {
  throw new SafeProbeFailure("invalid_utf8");
}
// after — tolerate compressed bodies
try {
  return new TextDecoder("utf-8", { fatal: true }).decode(bytes);
} catch {
  const decompressed = await new Response(new Blob([bytes]).stream().pipeThrough(new DecompressionStream("gzip"))).bytes();
  return new TextDecoder("utf-8", { fatal: true }).decode(decompressed);
}

const contentType = response.headers.get("content-type") ?? "";
if (!contentType.startsWith("text/") && !contentType.includes("json")) {
  throw new SafeProbeFailure("unexpected_content_type");
}

function isInvalidUtf8(e: unknown): e is SafeProbeFailure {
  return e instanceof SafeProbeFailure && e.code === "invalid_utf8";
}

try {
  const text = await readBoundedBody(response, deadline, budget);
} catch (error) {
  if (error instanceof SafeProbeFailure && error.code === "invalid_utf8") {
    return { ok: false, reason: "binary_response" };
  }
  throw error;
}

When it happens

Trigger: Probing an endpoint that returns compressed (gzip/br not auto-decompressed), encrypted, or otherwise binary bytes instead of UTF-8 text.

Common situations: Probe URL points at a binary download or the response arrives with content-encoding the fetch layer did not decode; server returns protobuf/msgpack instead of JSON.

Related errors

• response_too_large
• invalid_json
• ${ref.name} received malformed URI data: ${error instanceof

AI-assisted analysis of different-ai/openwork@2b7df46e8a (2026-09-01). Data as JSON: /api/errors/120a3fbed6647946. Report an issue: GitHub.