{"record":{"id":"b4cda64fa283d874","repo":"langchain-ai/langchain","slug":"nan-values-found-please-remove-the-nan-values-and","errorCode":null,"errorMessage":"NaN values found, please remove the NaN values and try again","messagePattern":"NaN values found, please remove the NaN values and try again","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"libs/core/langchain_core/vectorstores/utils.py","lineNumber":103,"sourceCode":"            f\"Number of columns in X and Y must be the same. X has shape {x.shape} \"\n            f\"and Y has shape {y.shape}.\"\n        )\n        raise ValueError(msg)\n    if not _HAS_SIMSIMD:\n        logger.debug(\n            \"Unable to import simsimd, defaulting to NumPy implementation. If you want \"\n            \"to use simsimd please install with `pip install simsimd`.\"\n        )\n        x_norm = np.linalg.norm(x, axis=1)\n        y_norm = np.linalg.norm(y, axis=1)\n        # Ignore divide by zero errors run time warnings as those are handled below.\n        with np.errstate(divide=\"ignore\", invalid=\"ignore\"):\n            similarity: npt.NDArray[np.floating] = np.dot(x, y.T) / np.outer(\n                x_norm, y_norm\n            )\n        if np.isnan(similarity).all():\n            msg = \"NaN values found, please remove the NaN values and try again\"\n            raise ValueError(msg) from None\n        similarity[np.isnan(similarity) | np.isinf(similarity)] = 0.0\n        return similarity\n\n    x = np.array(x, dtype=np.float32)\n    y = np.array(y, dtype=np.float32)\n    return 1 - np.array(simd.cdist(x, y, metric=\"cosine\"))\n\n\ndef maximal_marginal_relevance(\n    query_embedding: npt.NDArray[np.floating],\n    embedding_list: list[list[float]],\n    lambda_mult: float = 0.5,\n    k: int = 4,\n) -> list[int]:\n    \"\"\"Calculate maximal marginal relevance.\n\n    Args:\n        query_embedding: The query embedding.","sourceCodeStart":85,"sourceCodeEnd":121,"githubUrl":"https://github.com/langchain-ai/langchain/blob/e32fa9a52eab3b61ad7a45399bfde59b3e580fc4/libs/core/langchain_core/vectorstores/utils.py#L85-L121","documentation":"Raised by cosine_similarity (numpy path) when every entry of the computed similarity matrix is NaN. The numpy implementation normalizes by x_norm and y_norm; a zero-norm vector (all zeros) makes the division produce NaN/Inf entries. As a last line of defense the function checks np.isnan(similarity).all() — if nothing in the matrix is a valid number, no meaningful similarity exists and it raises rather than returning garbage. Note the check uses .all(): partially-NaN matrices do not raise; individual NaN/Inf entries are just zeroed.","triggerScenarios":"Passing at least one all-zero vector on one side while every pairing against the other side yields NaN — concretely, when a zero-norm row makes its entire row/column NaN and that covers the whole matrix (e.g. a single zero query vector against any documents, or zero vectors on both sides). Typical sources: an embedding model returning zeros for empty/whitespace text, dummy placeholder embeddings of [0.0]*dim, or a test fixture built with np.zeros.","commonSituations":"Embedding empty strings ('' or '   ') with models or mock embedders that return zero vectors; unit tests with placeholder zero embeddings passed through cosine_similarity; batch pipelines where a parsing bug produces empty documents whose embeddings are zeros; normalizing vectors to zero when a document's text is dropped.","solutions":["Find and fix the zero-norm inputs: filter or reject empty/whitespace texts before embedding (e.g. skip docs where not doc.page_content.strip()).","If a vector is legitimately all zeros in your pipeline, replace it with a small epsilon vector or drop that row before calling cosine_similarity.","Debug by locating zero rows: zero_rows = np.where(~np.any(x, axis=1))[0] (same for y) and inspecting the corresponding source texts.","Check your embedding call — a mock or offline stub returning [0.0]*dim will always trip this in tests; make stubs return distinct nonzero vectors."],"exampleFix":"// before\nquery_vec = embedder.embed_query(\"\")  # returns [0.0, 0.0, ..., 0.0]\nsim = cosine_similarity([query_vec], doc_vecs)  # all NaN -> ValueError\n\n// after\nquery = \"\"\nif not query.strip():\n    raise ValueError(\"query text is empty; refusing to embed\")\nquery_vec = embedder.embed_query(query)\nsim = cosine_similarity([query_vec], doc_vecs)","handlingStrategy":"validation","validationCode":"import numpy as np\n\ndef reject_zero_vectors(embeddings: list[list[float]]) -> None:\n    arr = np.asarray(embeddings, dtype=np.float32)\n    norms = np.linalg.norm(arr, axis=-1)\n    if np.any(norms == 0):\n        bad = np.where(norms == 0)[0].tolist()\n        msg = f\"zero-norm (all-zero) embeddings at indices {bad}; check empty inputs\"\n        raise ValueError(msg)","typeGuard":"import numpy as np\n\ndef all_vectors_nonzero(embeddings: object) -> bool:\n    \"\"\"True if input is array-like with no all-zero rows.\"\"\"\n    try:\n        arr = np.asarray(embeddings, dtype=np.float32)\n    except Exception:\n        return False\n    return bool(np.all(np.linalg.norm(arr, axis=-1) > 0))","tryCatchPattern":"try:\n    sim = cosine_similarity(x, y)\nexcept ValueError as e:\n    if \"NaN values found\" in str(e):\n        logger.warning(\"zero-norm vector in inputs; dropping it and retrying\")\n        x = [v for v in x if any(v)]\n        y = [v for v in y if any(v)]\n        sim = cosine_similarity(x, y)\n    else:\n        raise","preventionTips":["Reject empty or whitespace-only texts before embedding (if not text.strip(): skip or raise).","Make test/mock embedders return distinct nonzero vectors, never np.zeros.","Validate embeddings after generation: assert np.linalg.norm(vec) > 0.","Log a warning when an embedder returns identical zero vectors — it usually signals an upstream empty-input bug."],"tags":["validation","numpy","cosine-similarity","nan","zero-vector","embeddings"],"backgroundTag":null,"analyzedSha":"e32fa9a52eab3b61ad7a45399bfde59b3e580fc4","analyzedAt":"2026-08-14T18:42:09.092Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}