What causes "The system clock is not properly set" in databendlabs/databend?

Fix the system clock: enable NTP (chrony/systemd-timesyncd) and wait for sync before starting databend-query.

The system clock is not properly set

Error message

The system clock is not properly set

What it means

unix_ts() converts the current system time to a Duration since the UNIX epoch for session token claims. `SystemTime::now().duration_since(UNIX_EPOCH)` returns Err if the clock is set before 1970, so the expect panics. This guards token expire_at/issued-at computations from nonsense values.

Solutions

Fix the system clock: enable NTP (chrony/systemd-timesyncd) and wait for sync before starting databend-query.
Verify with `date -u` that the time is after 1970.
In containers/VMs, ensure the host clock is correct and RTC is set.
Optionally patch unix_ts to fall back to Duration::ZERO with a warning instead of panicking.

Defensive patterns

Strategy: validation

Validate before calling

# before starting databend-query
[ $(date -u +%s) -gt 0 ] || { echo "system clock is before UNIX epoch"; exit 1; }

Type guard

fn clock_is_sane() -> bool { std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).is_ok() }

Try / catch

let ts = SystemTime::now().duration_since(UNIX_EPOCH).unwrap_or_default();

Prevention

Enable NTP/time sync on all query nodes
Check `date -u` after VM/container boot before starting services
Set RTC/BIOS clock on bare metal
Alert on clock-skew monitoring

When it happens

Trigger: System clock set before 1970-01-01 (uninitialized RTC, wrong time in a fresh VM/container, NTP not yet synced) while issuing or refreshing session tokens.

Common situations: Embedded/fresh VMs booting with default BIOS time; containers started with a skewed clock; NTP sync delayed at startup; manually mis-set system date.

Understand the failure class

Background: "environment variable is not set" and "Missing keys in environment" errors: what missing required env var messages mean and how to fix them — this error's family across 28 libraries.

Related errors

AI-assisted analysis of databendlabs/databend@288d84d76e (2026-09-11). Data as JSON: /api/errors/9bca3d5ff46345a4. Report an issue: GitHub.

Appendix: source

Thrown at src/query/service/src/servers/http/v1/session/token.rs:49

    hex::encode(random_bytes)
}

#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct SessionClaim {
    pub tenant: String,
    pub user: String,
    pub auth_role: Option<String>,
    #[serde(rename = "sid")]
    pub session_id: String,
    pub(crate) nonce: String,
    #[serde(rename = "exp")]
    pub expire_at_in_secs: u64,
}

pub fn unix_ts() -> Duration {
    SystemTime::now()
        .duration_since(std::time::UNIX_EPOCH)
        .expect("The system clock is not properly set")
}

impl SessionClaim {
    pub fn new(
        session_id: String,
        tenant: &str,
        user: &str,
        auth_role: &Option<String>,
        ttl: Duration,
    ) -> Self {
        SessionClaim {
            tenant: tenant.to_string(),
            user: user.to_string(),
            auth_role: auth_role.clone(),
            session_id,
            nonce: generate_secure_nonce(),
            expire_at_in_secs: (unix_ts() + ttl).as_secs(),
        }

View on GitHub (pinned to 288d84d76e)