What causes "Kafka topic cannot be updated" in quickwit-oss/quickwit?

Create a new source (e.g. `quickwit source create`) pointing at the new topic instead of updating the existing one.

Kafka topic cannot be updated

Error message

Kafka topic cannot be updated

What it means

Kafka source params cannot have their `topic` changed via an update. Kafka partition IDs are stored as metastore checkpoint PartitionIds, and their uniqueness is not guaranteed across topics, so changing the topic would corrupt checkpoint semantics. `KafkaSourceParams::validate_update` rejects any update where the topic differs.

Source

Thrown at quickwit/quickwit-config/src/source_config/mod.rs:445

    #[serde(skip_serializing_if = "Option::is_none")]
    pub client_log_level: Option<String>,
    /// Kafka client configuration parameters.
    #[schema(value_type = Object)]
    #[serde(default = "serde_json::Value::default")]
    #[serde(skip_serializing_if = "serde_json::Value::is_null")]
    pub client_params: JsonValue,
    /// When backfill mode is enabled, the source exits after reaching the end of the topic.
    #[serde(default)]
    #[serde(skip_serializing_if = "is_false")]
    pub enable_backfill_mode: bool,
}

impl KafkaSourceParams {
    fn validate_update(&self, other: &Self) -> anyhow::Result<()> {
        // Updating the topic would likely mess up the checkpoints because the
        // Kafka partition IDs are used as metastore checkpoint PartitionId
        // and there uniqueness is not guaranteed across topics.
        ensure!(self.topic == other.topic, "Kafka topic cannot be updated");
        Ok(())
    }
}

#[derive(Clone, Debug, Eq, PartialEq, Hash, Serialize, Deserialize, utoipa::ToSchema)]
#[serde(deny_unknown_fields)]
pub struct PubSubSourceParams {
    /// Name of the subscription that the source consumes.
    pub subscription: String,
    /// When backfill mode is enabled, the source exits after reaching the end of the topic.
    #[serde(default)]
    #[serde(skip_serializing_if = "is_false")]
    pub enable_backfill_mode: bool,
    /// GCP service account credentials (`None` will use default via
    /// GOOGLE_APPLICATION_CREDENTIALS)
    /// Path to a google_cloud_auth::credentials::CredentialsFile serialized in JSON. See also
    /// `<https://cloud.google.com/docs/authentication/application-default-credentials>` and
    /// `<https://github.com/yoshidan/google-cloud-rust/tree/main/pubsub#automatically>` and

View on GitHub (pinned to a39730c5cd)

Solutions

Create a new source (e.g. `quickwit source create`) pointing at the new topic instead of updating the existing one.
If a topic rename is required: delete the old source and create a new one, accepting that ingestion restarts from the configured `default_start_timestamp`/beginning offsets.
If you only meant to change other params (bootstrap servers, consumer options), re-submit the update keeping `topic` identical.

Example fix

// before: PATCH existing source with changed topic
{"source_type":"kafka","params":{"topic":"logs-v2",...}}

// after: create a new source for the new topic
quickwit source create --index my-index --source-config kafka-logs-v2.json
// (kafka-logs-v2.json contains topic: logs-v2)

Defensive patterns

Strategy: validation

Validate before calling

let current = client.get_source(index_id, source_id).await?;
if current.params.topic != new_params.topic {
    return Err("Kafka topic is immutable; create a new source instead".into());
}
client.update_source(index_id, source_id, new_params).await?;

Try / catch

match client.update_source(index_id, source_id, params).await {
    Err(e) if e.to_string().contains("Kafka topic cannot be updated") => {
        // fall back to delete + create with the new topic
        client.delete_source(index_id, source_id).await?;
        client.create_source(index_id, source_config_with_new_topic).await?;
    }
    other => other?,
}

Prevention

Treat source_id + topic as one immutable identity; create a new source per topic.
Diff the submitted source config against the stored one before calling update.
Script topic migrations as delete+create, never update.

When it happens

Trigger: Calling the source update API/CLI (e.g. `quickwit source update`) with a Kafka source config whose `topic` field differs from the currently stored source config, triggering validate_update.

Common situations: Renaming or migrating a Kafka topic and trying to point the existing source at the new topic in place; copy-pasting a source config from another index; environment-specific configs (staging topic) applied to a production source.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/fb07ebd0b857045e. Report an issue: GitHub.