{"record":{"id":"3fb8fc334a5088f9","repo":"zed-industries/zed","slug":"streaming-not-supported-with-batching-client","errorCode":null,"errorMessage":"Streaming not supported with batching client","messagePattern":"Streaming not supported with batching client","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/edit_prediction_cli/src/anthropic_client.rs","lineNumber":836,"sourceCode":"\n    #[allow(dead_code)]\n    pub async fn generate_streaming<F>(\n        &self,\n        model: &str,\n        max_tokens: u64,\n        messages: Vec<Message>,\n        on_progress: F,\n    ) -> Result<Option<AnthropicResponse>>\n    where\n        F: FnMut(usize, &str),\n    {\n        match self {\n            AnthropicClient::Plain(plain_llm_client) => plain_llm_client\n                .generate_streaming(model, max_tokens, messages, on_progress)\n                .await\n                .map(Some),\n            AnthropicClient::Batch(_) => {\n                anyhow::bail!(\"Streaming not supported with batching client\")\n            }\n            AnthropicClient::Dummy => panic!(\"Dummy LLM client is not expected to be used\"),\n        }\n    }\n\n    pub async fn sync_batches(&self) -> Result<()> {\n        match self {\n            AnthropicClient::Plain(_) => Ok(()),\n            AnthropicClient::Batch(batching_llm_client) => batching_llm_client.sync_batches().await,\n            AnthropicClient::Dummy => panic!(\"Dummy LLM client is not expected to be used\"),\n        }\n    }\n\n    pub fn pending_batch_count(&self) -> Result<usize> {\n        match self {\n            AnthropicClient::Plain(_) => Ok(0),\n            AnthropicClient::Batch(batching_llm_client) => {\n                batching_llm_client.pending_batch_count()","sourceCodeStart":818,"sourceCodeEnd":854,"githubUrl":"https://github.com/zed-industries/zed/blob/f4178619acd0d47ea1f76a2025c42962c6d6638c/crates/edit_prediction_cli/src/anthropic_client.rs#L818-L854","documentation":"Thrown by AnthropicClient::generate_streaming when the client is the AnthropicClient::Batch variant. The enum wraps either a plain (interactive request/response) client or a batching client that only submits jobs to Anthropic's Batch API, which is inherently non-streaming — results arrive only after the whole batch completes. Calling a streaming API on it is a programming error, so it bails immediately.","triggerScenarios":"Constructing the CLI with a batching Anthropic client (e.g. a --provider that uses batch mode / --sync-batches flow) and then invoking generate_streaming(), which the Plain arm maps to streaming and the Batch arm rejects. The Dummy variant panics instead.","commonSituations":"Switching an edit_prediction_cli training/eval run from a non-batching teacher provider to a batching one while keeping a streaming progress callback path; refactoring call sites so a batch client leaks into a streaming code path.","solutions":["Use AnthropicClient::Plain (non-batching provider/flag) when you need streaming progress","Switch the call site to the non-streaming generate() plus sync_batches() flow for batch runs","Guard call sites with a variant check before choosing the streaming path (see typeGuard below)"],"exampleFix":"// before\nlet output = client.generate_streaming(model, max_tokens, messages, on_progress).await?;\n\n// after\nlet output = match &client {\n    AnthropicClient::Plain(_) => client.generate_streaming(model, max_tokens, messages, on_progress).await?,\n    _ => client.generate(model, max_tokens, messages).await?,\n};","handlingStrategy":"type-guard","validationCode":null,"typeGuard":"fn supports_streaming(client: &AnthropicClient) -> bool {\n    matches!(client, AnthropicClient::Plain(_))\n}","tryCatchPattern":"match client.generate_streaming(model, max_tokens, messages, on_progress).await {\n    Err(e) if e.to_string().contains(\"Streaming not supported\") => {\n        // fall back to the batch flow: generate + sync_batches\n        let out = client.generate(model, max_tokens, messages).await?;\n        client.sync_batches().await?;\n        Ok(Some(out))\n    }\n    other => other,\n}","preventionTips":["Pick the client variant from the requirement first: streaming -> Plain, batch throughput -> Batch","Branch on the enum variant before choosing the streaming code path instead of calling it unconditionally","Remember Batch has no incremental progress: design UIs/logs for batch runs without a progress callback"],"tags":["rust","anthropic","batching","streaming","api-misuse"],"backgroundTag":"unsupported-operation","analyzedSha":"f4178619acd0d47ea1f76a2025c42962c6d6638c","analyzedAt":"2026-08-20T19:29:52.058Z","contentChangedAt":"2026-08-20T19:29:52.058Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}