What causes "All of the configured Fulltext Search services failed." in phacility/phabricator?

Catch PhutilAggregateException and inspect `$ex->getExceptions()` — the first cause usually identifies the failing backend and reason.

phacility/phabricator · error · PhutilAggregateException

All of the configured Fulltext Search services failed.

Error message

All of the configured Fulltext Search services failed.

What it means

PhabricatorSearchService::newResultSet() tries each readable search service in turn and aggregates failures; if every readable service throws, it wraps the collected exceptions in a PhutilAggregateException with this message. Note that query-syntax errors (PhutilSearchQueryCompilerSyntaxException) are rethrown directly to the user and never reach this path — this error strictly means backend failure.

Source

Thrown at src/infrastructure/cluster/search/PhabricatorSearchService.php:276

      }

      try {
        $engine = $service->getEngine();
        $phids = $engine->executeSearch($query);

        return id(new PhabricatorFulltextResultSet())
          ->setPHIDs($phids)
          ->setFulltextTokens($engine->getFulltextTokens());
      } catch (PhutilSearchQueryCompilerSyntaxException $ex) {
        // If there's a query compilation error, return it directly to the
        // user: they issued a query with bad syntax.
        throw $ex;
      } catch (Exception $ex) {
        $exceptions[] = $ex;
      }
    }
    $msg = pht('All of the configured Fulltext Search services failed.');
    throw new PhutilAggregateException($msg, $exceptions);
  }

}

View on GitHub (pinned to 5720a38cfe)

Solutions

Catch PhutilAggregateException and inspect `$ex->getExceptions()` — the first cause usually identifies the failing backend and reason.
Restore backend availability (service up, index exists, `version`/`protocol`/`path` correct in `cluster.search`).
Add a second readable service (e.g. MySQL fulltext) so a single backend failure cannot take search down.
Run `bin/search status` to verify hosts report healthy.

Defensive patterns

Strategy: try-catch

Try / catch

try {
  $result_set = PhabricatorSearchService::newResultSet($query);
} catch (PhutilSearchQueryCompilerSyntaxException $ex) {
  // user syntax problem: surface directly
  throw $ex;
} catch (PhutilAggregateException $ex) {
  // all backends down: degrade, do not white-screen search pages
  foreach ($ex->getExceptions() as $cause) {
    phlog('search backend failure: '.$cause->getMessage());
  }
  return new PhabricatorFulltextResultSet()
    ->setPHIDs(array())
    ->setFulltextTokens(array());
}

Prevention

Run at least two readable search services (e.g. MySQL fulltext beside Elasticsearch) so aggregation has a survivor.
Alert on search health failure counts (`bin/search status`) so operators see degradation before total outage.
Distinguish syntax errors from availability errors in handlers — only the latter belong in this catch.

When it happens

Trigger: Executing a full-text search when all configured backends fail: Elasticsearch cluster down or unreachable, index deleted, authentication failures, or MySQL fulltext unavailable. Any single healthy readable service prevents the throw.

Common situations: Search outage during maintenance; all replicas marked read=false so only one backend serves reads and it fails; index deleted or recreated empty with a version mismatch.

Related errors

AI-assisted analysis of phacility/phabricator@5720a38cfe (2026-08-21). Data as JSON: /api/errors/7b3fc51df020bfee. Report an issue: GitHub.