weaviate/weaviate · warning

418 Maintenance mode

Error message

418 Maintenance mode

What it means

The clusterapi indices router returns HTTP 418 (Teapot) with body '418 Maintenance mode' when the node's maintenance-mode flag is enabled. All indices sub-handlers (search, find, aggregations, overwrite, digest) are short-circuited so replicated traffic stops during maintenance operations. The request itself is fine; the node is intentionally not serving.

Source

Thrown at adapters/handlers/rest/clusterapi/indices.go:233

		regexpShardReinit:                regexp.MustCompile(urlPatternShardReinit),
		regexpAsyncReplicationTargetNode: regexp.MustCompile(urlPatternAsyncReplicationTargetNode),
		shards:                           shards,
		db:                               db,
		auth:                             auth,
		maintenanceModeEnabled:           maintenanceModeEnabled,
		logger:                           logger,
	}
}

func (i *indices) Indices() http.Handler {
	return i.auth.handleFunc(i.indicesHandler())
}

func (i *indices) indicesHandler() http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		path := r.URL.Path
		if i.maintenanceModeEnabled() {
			http.Error(w, "418 Maintenance mode", http.StatusTeapot)
			return
		}
		// NOTE if you update any of these handler methods/paths, also update the indices_test.go
		// TestMaintenanceModeIndices test to include the new methods/paths.
		switch {
		case i.regexpObjectsSearch.MatchString(path):
			if r.Method != http.MethodPost {
				http.Error(w, "405 Method not Allowed", http.StatusMethodNotAllowed)
				return
			}

			i.postSearchObjects().ServeHTTP(w, r)
			return
		case i.regexpObjectsFind.MatchString(path):
			if r.Method != http.MethodPost {
				http.Error(w, "405 Method not Allowed", http.StatusMethodNotAllowed)
				return
			}

View on GitHub (pinned to 75aa4b6d11)

Solutions

  1. Retry the request against another node not in maintenance mode
  2. Disable maintenance mode (remove the maintenance-mode flag/config) and restart the node
  3. Wait for the maintenance window to complete before resuming replication traffic

Example fix

// before
resp, err := client.Get(maintenanceNodeURL + "/replication/indices/...")
// after
if nodeInMaintenance(maintenanceNodeURL) { resp, err = client.Get(healthyNodeURL + "/replication/indices/...") }
Defensive patterns

Strategy: retry

Validate before calling

// check node health/maintenance flag before sending replication traffic
resp, err := http.Get(nodeURL + "/_health/alive") // plus maintenance-flag check per deployment config

Try / catch

resp, err := client.Do(req)
if err != nil { return err }
if resp.StatusCode == http.StatusTeapot {
    // node in maintenance mode: fail over to another node, retry with backoff
    return failoverAndRetry(req)
}

Prevention

When it happens

Trigger: Any clusterapi indices request sent to a node that was started with maintenance mode enabled (e.g. during shard/offloading maintenance or node replacement).

Common situations: Rolling cluster maintenance; leftover maintenance-mode config flag after an operation finished; pointing client traffic at a node still flagged for maintenance.

Related errors


AI-assisted analysis of weaviate/weaviate@75aa4b6d11 (2026-09-04). Data as JSON: /api/errors/4c68ee9e61818a73. Report an issue: GitHub.