zed-industries/zed · error

failed to parse python semantic_token_rules.json

Error message

failed to parse python semantic_token_rules.json

What it means

Panics when the Python rules JSONC text fails to parse into SemanticTokenRules. Because file and type are versioned together, failure means structural drift between the vendored JSON and the Rust type (e.g., renamed fields), which is a developer error caught at first use of Python semantic tokens.

Source

Thrown at crates/languages/src/python.rs:60

use http_client::github_download::{GithubBinaryMetadata, download_server_binary};
use parking_lot::Mutex;
use std::str::FromStr;
use std::{
    borrow::Cow,
    fmt::Write,
    future::Future,
    path::{Path, PathBuf},
    sync::Arc,
};
use task::{ShellKind, TaskTemplate, TaskTemplates, VariableName};
use util::{ResultExt, maybe};

pub(crate) fn semantic_token_rules() -> SemanticTokenRules {
    let content = grammars::get_file("python/semantic_token_rules.json")
        .expect("missing python/semantic_token_rules.json");
    let json = std::str::from_utf8(&content.data).expect("invalid utf-8 in semantic_token_rules");
    settings::parse_json_with_comments::<SemanticTokenRules>(json)
        .expect("failed to parse python semantic_token_rules.json")
}

#[derive(Debug, Serialize, Deserialize)]
pub(crate) struct PythonToolchainData {
    #[serde(flatten)]
    environment: PythonEnvironment,
    #[serde(skip_serializing_if = "Option::is_none")]
    activation_scripts: Option<HashMap<ShellKind, PathBuf>>,
}

pub(crate) struct PyprojectTomlManifestProvider;

impl ManifestProvider for PyprojectTomlManifestProvider {
    fn name(&self) -> ManifestName {
        SharedString::new_static("pyproject.toml").into()
    }

    fn search(

View on GitHub (pinned to f4178619ac)

Solutions

1. Add a deserialization test for python/semantic_token_rules.json to surface the serde error
2. Re-sync the JSON file with the SemanticTokenRules type definition
3. Log and use default rules instead of panicking on parse failure

When it happens

Trigger: Thrown at crates/languages/src/python.rs:60 when the library encounters an invalid state.

Common situations: See trigger scenarios.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

AI-assisted analysis of zed-industries/zed@f4178619ac (2026-08-20). Data as JSON: /api/errors/cd58e76509e05dd9. Report an issue: GitHub.