phacility/phabricator · error · PhutilAggregateException

Writes to search services failed while reindexing document "

Error message

Writes to search services failed while reindexing document "%s".

What it means

PhabricatorSearchService::reindexAbstractDocument() writes the document to every writable search service and collects per-service exceptions instead of dropping them. If any writable engine throws (connection refused, mapping error, version mismatch, timeout), all failures are wrapped in a PhutilAggregateException carrying the document PHID, so callers can detect that the index is missing this document.

Source

Thrown at src/infrastructure/cluster/search/PhabricatorSearchService.php:234

  public static function reindexAbstractDocument(
    PhabricatorSearchAbstractDocument $document) {

    $exceptions = array();
    foreach (self::getAllServices() as $service) {
      if (!$service->isWritable()) {
        continue;
      }

      $engine = $service->getEngine();
      try {
        $engine->reindexAbstractDocument($document);
      } catch (Exception $ex) {
        $exceptions[] = $ex;
      }
    }

    if ($exceptions) {
      throw new PhutilAggregateException(
        pht(
          'Writes to search services failed while reindexing document "%s".',
          $document->getPHID()),
        $exceptions);
    }
  }

  /**
   * Execute a full-text query and return a list of PHIDs of matching objects.
   * @return string[]
   * @throws PhutilAggregateException
   */
  public static function executeSearch(PhabricatorSavedQuery $query) {
    $result_set = self::newResultSet($query);
    return $result_set->getPHIDs();
  }

  public static function newResultSet(PhabricatorSavedQuery $query) {

View on GitHub (pinned to 5720a38cfe)

Solutions

Catch PhutilAggregateException and iterate `$ex->getExceptions()` to see which backend failed and why (connection vs mapping vs version).
Restore the failing backend, then re-run the reindex — indexing is idempotent and will fill the gap.
If mappings changed, recreate the index and reindex from scratch rather than patching documents.
Temporarily mark failing hosts non-writable via their `roles` map so writes stop routing to a dead backend while you repair it.

Defensive patterns

Strategy: try-catch

Try / catch

try {
  PhabricatorSearchService::reindexAbstractDocument($document);
} catch (PhutilAggregateException $ex) {
  foreach ($ex->getExceptions() as $cause) {
    phlog('search reindex failure: '.$cause->getMessage());
  }
  // indexing is idempotent: queue the PHID for a later retry pass
  // rather than failing the whole job
}

Prevention

Treat reindex failures as retryable background work (queue the PHID), not fatal errors in the request path.
Keep `bin/search index` in periodic scheduled runs so gaps from transient outages self-heal.
Mark known-dead hosts non-writable via roles instead of letting every write attempt fail and aggregate.

When it happens

Trigger: Running `bin/search index` or saving/creating objects while a writable backend fails: Elasticsearch node down, index mapping changed after a version bump, wrong `version`/`protocol`/`path` in `cluster.search`, or network timeouts. Each failing writable service contributes its exception to the aggregate.

Common situations: Reindexing after upgrading the search backend (mapping incompatibilities); network partition between app hosts and the search cluster; credentials or index permissions changed mid-reindex.

Related errors

AI-assisted analysis of phacility/phabricator@5720a38cfe (2026-08-21). Data as JSON: /api/errors/1ce3eb0a0773ec09. Report an issue: GitHub.