Netflix/chaosmonkey · error
deploySchedule: could not publish schedule: %v
Error message
deploySchedule: could not publish schedule: %v
What it means
After computing 'today' in the local timezone, deploySchedule persists the schedule via ss.Publish (the SchedStore, e.g. backed by Cloud Datastore or chaosmonkey-api). If Publish returns an error, the schedule was not saved and this wrapped error is returned; cron registration is skipped. The underlying cause follows the colon.
Source
Thrown at command/schedule.go:91
}
return nil
}
// deploySchedule publishes the schedule to chaosmonkey-api
// and registers the schedule with the local cron
func deploySchedule(s *schedule.Schedule, ss schedstore.SchedStore, cfg *config.Monkey) error {
loc, err := cfg.Location()
if err != nil {
return fmt.Errorf("deploySchedule: could not retrieve local timezone: %v", err)
}
today := time.Now().In(loc)
err = ss.Publish(today, s)
if err != nil {
return fmt.Errorf("deploySchedule: could not publish schedule: %v", err)
}
err = registerWithCron(s, cfg)
return err
}
// registerWithCron registers the schedule of terminations with cron on the local machine
//
// Creates or overwrites the file specified by config.Chaos.CronPath()
func registerWithCron(s *schedule.Schedule, cfg *config.Monkey) error {
crontab := s.Crontab(cfg.TermPath(), cfg.TermAccount())
var perms os.FileMode = 0644 // -rw-r--r--
log.Printf("Writing %s\n", cfg.CronPath())
err := ioutil.WriteFile(cfg.CronPath(), crontab, perms)
return err
}
View on GitHub (pinned to eaa28fb761)
Solutions
- Read the wrapped cause to identify whether it is a network, auth, or storage error.
- Verify the schedstore backend (Datastore/chaosmonkey-api) is up and credentials have write access to the schedule entities.
- Check network connectivity from the monkey host to the storage backend and retry after transient failures.
- Validate the generated schedule entries if the cause indicates a bad-entity/payload error.
Defensive patterns
Strategy: retry
Validate before calling
// preflight: confirm the schedstore backend is writable before deploying
if err := schedStoreHealthy(ss); err != nil {
return fmt.Errorf("schedstore unavailable: %v", err)
}
// schedStoreHealthy issues a lightweight read/write probe per your SchedStore implementation Try / catch
if err := do(dep, appCfgGetter, schedStore, cfg, constrainer, apps); err != nil {
if strings.Contains(err.Error(), "could not publish schedule") {
log.Errorf("publish failed: %v", errors.Unwrap(err))
// retry with backoff for transient backend errors
for i := 0; i < 3; i++ {
time.Sleep(time.Duration(1<<i) * time.Second)
if err2 := do(dep, appCfgGetter, schedStore, cfg, constrainer, apps); err2 == nil {
return nil
}
}
return err
}
return err
} Prevention
- Monitor and alert on schedstore (Datastore/API) availability before scheduling windows.
- Grant the service account explicit write permissions to schedule entities.
- Implement bounded retries with exponential backoff for transient publish failures.
- Keep schedule payloads within store limits and validate entries before publishing.
When it happens
Trigger: deploySchedule (via the schedule command) when ss.Publish(today, s) fails: storage backend unreachable, authentication/permission failure, or the store rejects the schedule payload.
Common situations: Cloud Datastore/API outages, missing service-account permissions to write schedule entities, network partition from the monkey server, or corrupted/oversized schedule data.
Related errors
AI-assisted analysis of Netflix/chaosmonkey@eaa28fb761 (2026-09-03).
Data as JSON: /api/errors/29524745a512afe4.
Report an issue: GitHub.