sgl-project/sglang · error · Error

/v1/models ${response.status}

Error message

/v1/models ${response.status}

What it means

RadixTree::writing_through in the tree_v2 C++ radix tree is only implemented for the hicache-disabled fast path; when hierarchical caching is enabled (use_hicache true) the write-through-to-host path is a stub that throws 'Not implemented yet'. It exists so the API surface compiles while the IO path is pending.

Source

Thrown at python/sglang/multimodal_gen/apps/realtime_webui/app.js:1724

  return String(info?.id || info?.model || info?.root || "");
}

function presetForModelInfo(info) {
  const id = servedModelId(info).toLowerCase();
  if (!id) return null;
  return presets.find((preset) => (
    preset.model && id.includes(preset.model.toLowerCase())
  )) || null;
}

async function queryServerModelInfo(options = {}) {
  const applyPresetForModel = options.applyPresetForModel ?? true;
  let info;
  try {
    const response = await fetch(modelsUrlFromServerUrl($("serverUrl").value), {
      cache: "no-store",
    });
    if (!response.ok) throw new Error(`/v1/models ${response.status}`);
    info = firstServedModelInfo(await response.json());
  } catch (error) {
    addHistory(`model query failed · ${error.message || "unknown"}`);
    return null;
  }
  if (!info) return null;

  const modelId = servedModelId(info);
  const preset = presetForModelInfo(info);
  if (preset && applyPresetForModel && preset !== selectedPreset) {
    await applyPreset(preset, { sendRuntimeEvents: false });
  }
  if (modelId) $("model").value = modelId;
  addHistory(
    preset
      ? `server model · ${preset.name}`
      : `server model · ${modelId || "unknown"}`,
  );

View on GitHub (pinned to 0132848349)

Solutions

1. Disable hierarchical cache (use the non-hicache configuration) with tree_v2 — the disabled path returns cleanly
2. Fall back to the original/v1 cpp radix tree or Python radix cache backend which implements write-through
3. Track the upstream PR implementing tree_v2 hicache IO and upgrade once merged

Example fix

# before
server_args.enable_hicache = True  # + tree_v2 backend -> throws

# after
server_args.enable_hicache = False  # with tree_v2, or keep hicache and use v1 radix tree

from sglang.srt.mem_cache.cpp_radix_tree import RadixTree  # tree_v2
# before enabling hicache with tree_v2:
assert not use_hicache, "tree_v2 writing_through is unimplemented when hicache is on"

def tree_v2_supports_hicache(tree) -> bool:
    return not getattr(tree, "is_v2", False) or not tree.impl.use_hicache

try:
    tree.writing_through(node, kvs)
except RuntimeError as e:
    if "Not implemented yet" in str(e):
        logger.warning("tree_v2 hicache write-through unsupported; skipping")
    else:
        raise

When it happens

Trigger: Enabling HiCache (use_hicache) with the tree_v2 cpp radix tree backend and triggering a write-through of a newly inserted prefix — i.e. any cache insert once hierarchical caching is active on this tree version.

Common situations: Switching to the experimental tree_v2 backend while running with --enable-hicache or hierarchical cache config; upgrading SGLang where tree_v2 advertises the API but the host-tier IO is unfinished.

Related errors

• Unsupported msgpack byte ${b}
• This browser does not support gzip stream decoding
• Ascend PD transfer does not support HiSparse destination dev
• get_split_heads_page_buffer_meta requires layout='page_head'
• Host cache is not supported yet

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/16415dad5b56b11e. Report an issue: GitHub.