JuliusBrussee/caveman · error · ReplayRunError

cachebench: request %q exceeded schedule drift tolerance

Error message

cachebench: request %q exceeded schedule drift tolerance

What it means

A request was actually sent later than its scheduled time by more than the tolerated drift (ScheduleDriftMilliseconds exceeded the allowance under RequireGroundedTiming), so the replay's timing fidelity claim would be false.

Source

Thrown at cacheengine/cachebench/replay.go:578

		traceAt, _ := time.Parse(time.RFC3339Nano, record.At)
		offset, _ := scaledReplayGap(traceAt.Sub(anchorTrace), runner.TimeScale)
		scheduled := anchorReal.Add(offset)
		if delay := scheduled.Sub(now().UTC()); delay > 0 {
			if err := sleep(ctx, delay); err != nil {
				return fmt.Errorf("cachebench: replay schedule interrupted: %w", err)
			}
		}
		started := now().UTC()
		evidence := replayEvidenceBase(record, item.optimized, runner.TimeScale, scheduled, started, runner.Limits.MaxScheduleDrift)
		if runner.Limits.RequireGroundedTiming && evidence.ScheduleDriftMilliseconds > evidence.ScheduleToleranceMilliseconds {
			evidence.FailureCode = "schedule_drift"
			evidence.CompletedAt = now().UTC().Format(time.RFC3339Nano)
			if err := emitValidatedReplayResult(emit, ReplayResult{Evidence: evidence}); err != nil {
				return err
			}
			return &ReplayRunError{
				RequestID: record.RequestID, FailureCode: evidence.FailureCode,
				Err: fmt.Errorf("cachebench: request %q exceeded schedule drift tolerance", record.RequestID),
			}
		}
		result, runErr := runner.executePrepared(ctx, item, evidence, started, now)
		if err := emitValidatedReplayResult(emit, result); err != nil {
			return err
		}
		if runErr != nil {
			return runErr
		}
	}
	return nil
}

func (runner ReplayRunner) runConcurrent(ctx context.Context, prepared []preparedReplay, anchorTrace, anchorReal time.Time, now func() time.Time, sleep func(context.Context, time.Duration) error, emit func(ReplayResult) error) error {
	scheduleCtx, stopScheduling := context.WithCancel(ctx)
	defer stopScheduling()
	semaphore := make(chan struct{}, runner.Limits.MaxConcurrency)
	var workers sync.WaitGroup

View on GitHub (pinned to 766dce6b13)

Solutions

1. Raise MaxScheduleDrift tolerance
2. Reduce concurrency or load so sends meet their schedule

When it happens

Trigger: Thrown at cacheengine/cachebench/replay.go:578 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of JuliusBrussee/caveman@766dce6b13 (2026-08-18). Data as JSON: /api/errors/6b705910e0d6eaed. Report an issue: GitHub.