apache/dolphinscheduler · error · CodeGenerateException

New code exception because time is set back.

Error message

New code exception because time is set back.

What it means

CodeGenerateUtils.genCode implements a Snowflake-style ID generator that guarantees monotonically increasing IDs per instance. If the current system millisecond is earlier than the last recorded millisecond (recordMillisecond), the generator would risk issuing duplicate codes, so it throws CodeGenerateException with this message.

Source

Thrown at dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/CodeGenerateUtils.java:57

        private static final long MACHINE_BIT = 5L;
        private static final long MAX_LOW_DIGIT = ~(-1L << LOW_DIGIT_BIT);
        // The displacement to the left
        private static final long HIGH_DIGIT_LEFT = LOW_DIGIT_BIT + MACHINE_BIT;
        public final long machineHash;
        private long lowDigit = 0L;
        private long recordMillisecond = -1L;

        private static final long SYSTEM_TIMESTAMP = System.currentTimeMillis();
        private static final long SYSTEM_NANOTIME = System.nanoTime();

        public CodeGenerator(String appName) {
            this.machineHash = Math.abs(Objects.hash(appName)) % (1 << MACHINE_BIT);
        }

        public synchronized long genCode() throws CodeGenerateException {
            long nowtMillisecond = systemMillisecond();
            if (nowtMillisecond < recordMillisecond) {
                throw new CodeGenerateException("New code exception because time is set back.");
            }
            if (nowtMillisecond == recordMillisecond) {
                lowDigit = (lowDigit + 1) & MAX_LOW_DIGIT;
                if (lowDigit == 0L) {
                    while (nowtMillisecond <= recordMillisecond) {
                        nowtMillisecond = systemMillisecond();
                    }
                }
            } else {
                lowDigit = 0L;
            }
            recordMillisecond = nowtMillisecond;
            return (nowtMillisecond - START_TIMESTAMP) << HIGH_DIGIT_LEFT | machineHash << LOW_DIGIT_BIT | lowDigit;
        }

        private long systemMillisecond() {
            return SYSTEM_TIMESTAMP + (System.nanoTime() - SYSTEM_NANOTIME) / 1000000;
        }

View on GitHub (pinned to 02eac45a1b)

Solutions

1. Wait until the system clock catches back up to recordMillisecond (plus a small safety margin) and retry
2. Sync the clock forward via NTP (chronyd/ntpd) or restart the service so the generator re-initializes recordMillisecond from the current clock
3. Fix the underlying clock source: ensure the VM/host has reliable NTP and disable backwards clock stepping (use slewing instead of stepping)
4. If reproducible in code, construct the CodeGenerator after the clock is stable; each new instance resets recordMillisecond

Example fix

// before
long code = codeGenerateUtils.genCode(); // throws after NTP step-back

// after
try {
    long code = codeGenerateUtils.genCode();
} catch (CodeGenerateException e) {
    // clock moved backwards; wait for clock to catch up then retry
    Thread.sleep(100);
    long code = codeGenerateUtils.genCode();
}

// check clock monotonicity before generating
long now = System.currentTimeMillis();
if (now < lastSeenMillis) {
    throw new IllegalStateException("Clock moved backwards by " + (lastSeenMillis - now) + "ms; fix NTP before generating IDs");
}

try {
    code = codeGenerator.genCode();
} catch (CodeGenerateException e) {
    if (e.getMessage().contains("time is set back")) {
        Thread.sleep(50); // wait for clock to catch up
        code = codeGenerator.genCode();
    } else {
        throw e;
    }
}

When it happens

Trigger: genCode() is called after the system clock moved backwards: NTP correction after drift, VM live-migration/resume restoring an older clock, manual clock adjustment, or container clock skew relative to a previous process run.

Common situations: NTP daemon stepping the clock back on a worker node; VM snapshots resumed on a host with a lagging clock; Docker/Kubernetes containers whose clock resyncs backwards after host sleep; running tests that manipulate the clock (the caller testNoGenerateDuplicateCodeWithDifferentAppName exercises the generator directly).

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/4449b8cfbefb964e. Report an issue: GitHub.