pathwaycom/pathway · error · NotImplementedError

Currently, tantivy bm25 index is supported only in the as-of

Error message

Currently, tantivy bm25 index is supported only in the as-of-now variant

What it means

Pathway's BM25FullTextIndex (tantivy-backed) currently implements only query_as_of_now, i.e. queries evaluate against the index state as of the current minibatch without waiting for pending updates. The general query() method is declared for interface parity with other indices but unconditionally raises NotImplementedError with this message.

Source

Thrown at python/pathway/stdlib/indexing/bm25.py:67

            string representation of some auxiliary data, in JSON format.
        ram_budget (int): maximum capacity in bytes. When reached, the index moves a block of data
            to storage (hence, larger budget means faster index operations, but higher
            memory cost)
        in_memory_index (bool): indicates, whether the whole index is stored in RAM;
            if set to false, the index is stored in some default Pathway Live Data Framework disk storage
    """

    ram_budget: int = 50 * 1024 * 1024  # 50 MB
    in_memory_index: bool = True

    def query(
        self,
        query_column: pw.ColumnReference,
        number_of_matches: pw.ColumnExpression | int = 3,
        metadata_filter: pw.ColumnExpression | None = None,
    ) -> pw.Table:
        """Currently, tantivy bm25 index is supported only in the as-of-now variant"""
        raise NotImplementedError(
            "Currently, tantivy bm25 index is supported only in the as-of-now variant"
        )

    @check_arg_types
    def query_as_of_now(
        self,
        query_column: pw.ColumnReference,
        number_of_matches: pw.ColumnExpression | int = 3,
        metadata_filter: pw.ColumnExpression | None = None,
    ) -> pw.Table:
        check_default_bm25_column_types(
            self.data_column,
            query_column,
            number_of_matches,
            self.metadata_column,
            metadata_filter,
        )

View on GitHub (pinned to fa2f74a464)

Solutions

Call query_as_of_now(query_column, number_of_matches=..., metadata_filter=...) instead of query().
If the call site is generic over index types, branch on availability or use getattr(index, 'query_as_of_now') when query() raises NotImplementedError.
Watch Pathway release notes — full query() support for tantivy BM25 is a known roadmap item.

Example fix

# before
res = index.query(pw.this.query, number_of_matches=5)

# after
res = index.query_as_of_now(pw.this.query, number_of_matches=5)

Defensive patterns

Strategy: type-guard

Type guard

def supports_blocking_query(index) -> bool:
    """False for indices whose query() raises NotImplementedError."""
    try:
        import inspect
        src = inspect.getsource(type(index).query)
        return "NotImplementedError" not in src
    except (OSError, TypeError):
        return True

Try / catch

try:
    res = index.query(q)
except NotImplementedError:
    res = index.query_as_of_now(q)

Prevention

Standardize retrieval code on query_as_of_now across all Pathway indices.
Pin the Pathway version and re-check index capabilities when upgrading, since query() support may be added later.

When it happens

Trigger: Calling .query(query_column, ...) on an index built by pw.ml.indexing.bm25.BM25FullTextIndex (or its factory) instead of .query_as_of_now().

Common situations: Code written against a different Pathway index (e.g. vector KNN indices or a generic Retriever wrapper) calling .query() polymorphically; older examples/tutorials using .query(); LLM xpack retrievers that default to the non-as-of-now path.

Related errors

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/7fe892d815d2821e. Report an issue: GitHub.