{"record":{"id":"7fe892d815d2821e","repo":"pathwaycom/pathway","slug":"currently-tantivy-bm25-index-is-supported-only-in","errorCode":null,"errorMessage":"Currently, tantivy bm25 index is supported only in the as-of-now variant","messagePattern":"Currently, tantivy bm25 index is supported only in the as-of-now variant","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/pathway/stdlib/indexing/bm25.py","lineNumber":67,"sourceCode":"            string representation of some auxiliary data, in JSON format.\n        ram_budget (int): maximum capacity in bytes. When reached, the index moves a block of data\n            to storage (hence, larger budget means faster index operations, but higher\n            memory cost)\n        in_memory_index (bool): indicates, whether the whole index is stored in RAM;\n            if set to false, the index is stored in some default Pathway Live Data Framework disk storage\n    \"\"\"\n\n    ram_budget: int = 50 * 1024 * 1024  # 50 MB\n    in_memory_index: bool = True\n\n    def query(\n        self,\n        query_column: pw.ColumnReference,\n        number_of_matches: pw.ColumnExpression | int = 3,\n        metadata_filter: pw.ColumnExpression | None = None,\n    ) -> pw.Table:\n        \"\"\"Currently, tantivy bm25 index is supported only in the as-of-now variant\"\"\"\n        raise NotImplementedError(\n            \"Currently, tantivy bm25 index is supported only in the as-of-now variant\"\n        )\n\n    @check_arg_types\n    def query_as_of_now(\n        self,\n        query_column: pw.ColumnReference,\n        number_of_matches: pw.ColumnExpression | int = 3,\n        metadata_filter: pw.ColumnExpression | None = None,\n    ) -> pw.Table:\n        check_default_bm25_column_types(\n            self.data_column,\n            query_column,\n            number_of_matches,\n            self.metadata_column,\n            metadata_filter,\n        )\n","sourceCodeStart":49,"sourceCodeEnd":85,"githubUrl":"https://github.com/pathwaycom/pathway/blob/fa2f74a4649b7c5908690cf60137263d8d80de5f/python/pathway/stdlib/indexing/bm25.py#L49-L85","documentation":"Pathway's BM25FullTextIndex (tantivy-backed) currently implements only query_as_of_now, i.e. queries evaluate against the index state as of the current minibatch without waiting for pending updates. The general query() method is declared for interface parity with other indices but unconditionally raises NotImplementedError with this message.","triggerScenarios":"Calling .query(query_column, ...) on an index built by pw.ml.indexing.bm25.BM25FullTextIndex (or its factory) instead of .query_as_of_now().","commonSituations":"Code written against a different Pathway index (e.g. vector KNN indices or a generic Retriever wrapper) calling .query() polymorphically; older examples/tutorials using .query(); LLM xpack retrievers that default to the non-as-of-now path.","solutions":["Call query_as_of_now(query_column, number_of_matches=..., metadata_filter=...) instead of query().","If the call site is generic over index types, branch on availability or use getattr(index, 'query_as_of_now') when query() raises NotImplementedError.","Watch Pathway release notes — full query() support for tantivy BM25 is a known roadmap item."],"exampleFix":"# before\nres = index.query(pw.this.query, number_of_matches=5)\n\n# after\nres = index.query_as_of_now(pw.this.query, number_of_matches=5)","handlingStrategy":"type-guard","validationCode":null,"typeGuard":"def supports_blocking_query(index) -> bool:\n    \"\"\"False for indices whose query() raises NotImplementedError.\"\"\"\n    try:\n        import inspect\n        src = inspect.getsource(type(index).query)\n        return \"NotImplementedError\" not in src\n    except (OSError, TypeError):\n        return True","tryCatchPattern":"try:\n    res = index.query(q)\nexcept NotImplementedError:\n    res = index.query_as_of_now(q)","preventionTips":["Standardize retrieval code on query_as_of_now across all Pathway indices.","Pin the Pathway version and re-check index capabilities when upgrading, since query() support may be added later."],"tags":["pathway","bm25","full-text-search","not-implemented","api-limitation"],"backgroundTag":null,"analyzedSha":"fa2f74a4649b7c5908690cf60137263d8d80de5f","analyzedAt":"2026-08-15T01:48:17.006Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}