What causes "err.to_string()" in quickwit-oss/quickwit?

Retry the call — the function recomputes `upcoming(Utc)` each time, so a transient race resolves on the next call.

err.to_string()

Error message

err.to_string()

What it means

RetentionConfig's `duration_until_next_evaluation` computes the time until the schedule's next occurrence using cron `Schedule::upcoming(Utc)`. The difference between the next date and now is converted to a `std::time::Duration`; if the next date is in the past (negative duration), `to_std()` fails and its Display is wrapped into this anyhow error.

Source

Thrown at quickwit/quickwit-config/src/index_config/mod.rs:371

        let evaluation_schedule = prepend_at_char(&self.evaluation_schedule);

        Schedule::from_str(&evaluation_schedule).with_context(|| {
            format!(
                "failed to parse retention evaluation schedule `{}`",
                self.evaluation_schedule
            )
        })
    }

    pub fn duration_until_next_evaluation(&self) -> anyhow::Result<Duration> {
        let schedule = self.evaluation_schedule()?;
        let mut schedule_iter = schedule.upcoming(Utc);
        let future_date = schedule_iter
            .next()
            .expect("Failed to obtain next evaluation date.");
        let mut duration = (future_date - Utc::now())
            .to_std()
            .map_err(|err| anyhow::anyhow!(err.to_string()))?;
        let jitter_max_secs = self
            .evaluation_schedule_jitter
            .as_deref()
            .copied()
            .unwrap_or_else(|| {
                if let Some(next_next_date) = schedule_iter.next() {
                    let time_between_schedules = next_next_date - future_date;
                    Duration::from_secs(time_between_schedules.num_seconds().clamp(0, 3600) as u64)
                } else {
                    // we don't know when the schedule is. That's odd. Let's allow no jitter
                    warn!("found retention policy schedule with no next execution");
                    Duration::ZERO
                }
            })
            .as_secs();
        let jitter = rng().sample::<u64, _>(distr::StandardUniform) % (jitter_max_secs + 1);
        duration += Duration::from_secs(jitter);
        Ok(duration)

View on GitHub (pinned to a39730c5cd)

Solutions

Retry the call — the function recomputes `upcoming(Utc)` each time, so a transient race resolves on the next call.
Widen the cron interval or add jitter (`evaluation_schedule_jitter`) so ticks are not sub-second tight against evaluation latency.
Check system clock sync (NTP) if the machine's clock is jumping; verify the container/host time matches UTC expectations.
If writing tests, use a schedule comfortably in the future or inject a controlled clock.

Example fix

// before
let duration = (future_date - Utc::now()).to_std().map_err(|err| anyhow::anyhow!(err.to_string()))?;
// after: clamp so a just-passed tick yields zero duration instead of an error
let duration = (future_date - Utc::now()).to_std().unwrap_or(std::time::Duration::ZERO);

Defensive patterns

Strategy: retry

Validate before calling

// No pre-call validation possible; ensure host clock is correct:
// timedatectl | grep -i synchronized  (expect 'synchronized: yes')

Try / catch

match duration_until_next_evaluation(config) {
    Ok(d) => schedule_after(d),
    Err(e) if e.to_string().contains("out of range") || e.to_string().contains("earlier") => {
        // transient clock race: retry after a short backoff
        tokio::time::sleep(Duration::from_secs(1)).await;
        // retry
    }
    Err(e) => return Err(e),
}

Prevention

Keep NTP/time sync enabled on hosts and containers.
Avoid cron schedules with sub-second granularity for retention evaluation.
Set evaluation_schedule_jitter so evaluations are not perfectly aligned with schedule ticks.
In tests, use schedules comfortably in the future or inject a controllable clock.

When it happens

Trigger: Calling duration_until_next_evaluation on a retention policy whose cron schedule's next occurrence resolves to a timestamp before now — typically a clock race: the iterator was fetched, then time advanced past `future_date` before subtraction, or the schedule produced a date <= now.

Common situations: Very frequent cron schedules (e.g. every second) racing with slow evaluation; system clock adjustments (NTP corrections, container clock skew); calling the function in tests with a schedule whose next tick already elapsed.

Related errors

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/9f6649cf73157c02. Report an issue: GitHub.