quickwit-oss/quickwit · error

compactor is enabled but no janitor node was found in the cl

Error message

compactor is enabled but no janitor node was found in the cluster

What it means

When the compactor role is enabled, Quickwit's server discovers a janitor node hosting the remote CompactionPlannerService via gRPC balance-channel service discovery. If no janitor connection appears within COMPACTION_SERVICE_DISCOVERY_TIMEOUT, serve_quickwit bails with this message instead of running compaction without a planner.

Source

Thrown at quickwit/quickwit-serve/src/lib.rs:312

    }
    if is_janitor {
        let planner = CompactionPlanner::new(metastore_client.clone(), cluster.clone());
        let (mailbox, handle) = universe.spawn_builder().spawn(planner);
        info!("compaction planner actor started on janitor node");
        return Ok((
            Some(CompactionPlannerServiceClient::from_mailbox(mailbox)),
            Some(handle),
        ));
    }
    // Compactor-only node: connect to the planner on a remote janitor.
    let balance_channel = balance_channel_for_service(cluster, QuickwitService::Janitor).await;
    let found = balance_channel
        .wait_for(COMPACTION_SERVICE_DISCOVERY_TIMEOUT, |connections| {
            !connections.is_empty()
        })
        .await;
    if !found {
        bail!("compactor is enabled but no janitor node was found in the cluster")
    }
    info!("remote compaction planner detected on janitor node");
    Ok((
        Some(CompactionPlannerServiceClient::from_balance_channel(
            balance_channel,
            node_config.grpc_config.max_message_size,
            None,
        )),
        None,
    ))
}

fn spawn_merge_scheduler_service(
    universe: &Universe,
    node_config: &NodeConfig,
) -> Mailbox<MergeSchedulerService> {
    let (mailbox, _) = universe.spawn_builder().spawn(MergeSchedulerService::new(
        node_config.indexer_config.merge_concurrency.get(),

View on GitHub (pinned to a39730c5cd)

Solutions

1. Enable the janitor role in the cluster config so at least one node advertises it (or run janitor on the same node in single-node setups).
2. Verify the janitor node is running and its gRPC port is reachable (curl/nc the address, check Chitchat cluster member list).
3. Wait for/restart the janitor until it registers with the cluster, then retry the compactor start.
4. If remote compaction is not wanted, disable the compactor role instead of leaving it enabled without a janitor.

Example fix

// before (node_config: compactor enabled, janitor missing)
[compactor]
enable = true
// after (add janitor role to the cluster)
[janitor]
enable = true
[compactor]
enable = true

// Before starting the compactor, verify a janitor is advertised in the cluster
const members = await fetch('http://node:7280/health/livez') &&
  await fetchClusterMembers(); // e.g. via Chitchat/admin endpoint
if (!members.some(m => m.roles.includes('janitor'))) {
  throw new Error('Refusing to enable compactor: no janitor node in cluster');
}

function hasJanitor(members) {
  return Array.isArray(members) &&
    members.some(m => Array.isArray(m.roles) && m.roles.includes('janitor'));
}

try {
  await startQuickwitNode(config);
} catch (e) {
  if (e.message.includes('no janitor node was found')) {
    console.error('Enable the janitor role on a cluster node or disable the compactor.');
  }
  process.exit(1);
}

When it happens

Trigger: Starting a node with the compactor enabled in the config while (a) no node in the cluster runs the janitor role, (b) the janitor node is down or unreachable via gRPC, or (c) the janitor has not yet registered its gRPC service in Chitchat within the discovery timeout window.

Common situations: Misconfigured cluster where roles are split but the janitor role was omitted; janitor deployment crashed or scaled to 0; networking/firewall blocking the gRPC port; single-node setups enabling compactor but not janitor; transient startup ordering where compactor starts before the janitor registers.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• could not find control plane in the cluster
• could not find any `{service}` node in the cluster
• timed out while waiting for ingester to transition to status
• position of a Kafka partition should never be EOF
• position of a Kinesis shard should never be EOF

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/5e117d3252f4649c. Report an issue: GitHub.