{"record":{"id":"da2e90a78ca42a7e","repo":"MemPalace/mempalace","slug":"embedding-api-at-self-url-returned-non-contiguo","errorCode":null,"errorMessage":"Embedding API at {self._url} returned non-contiguous or duplicate 'index' values; cannot align embeddings with inputs","messagePattern":"Embedding API at (.+?) returned non-contiguous or duplicate 'index' values; cannot align embeddings with inputs","errorType":"exception","errorClass":"EmbeddingAPIError","httpStatus":null,"severity":"error","filePath":"mempalace/embedding.py","lineNumber":604,"sourceCode":"                f\"Embedding API at {self._url} returned no 'data' array: {data.get('error', data)}\"\n            )\n        if len(rows) != n:\n            raise EmbeddingAPIError(\n                f\"Embedding API at {self._url} returned {len(rows)} embeddings for {n} inputs\"\n            )\n        # The endpoint may return rows out of order — sort by index, then\n        # require the indices to be exactly 0..n-1 so positional alignment is\n        # provably correct (a server using absolute or duplicate indices would\n        # otherwise pass the count check yet map vectors to the wrong texts).\n        try:\n            rows = sorted(rows, key=lambda d: d.get(\"index\", -1))\n            indices = [r.get(\"index\") for r in rows]\n        except AttributeError as e:\n            raise EmbeddingAPIError(\n                f\"Embedding API at {self._url} returned non-object rows: {e}\"\n            ) from e\n        if indices != list(range(n)):\n            raise EmbeddingAPIError(\n                f\"Embedding API at {self._url} returned non-contiguous or duplicate \"\n                f\"'index' values; cannot align embeddings with inputs\"\n            )\n        try:\n            arr = np.asarray([r[\"embedding\"] for r in rows], dtype=np.float32)\n        except (KeyError, TypeError, ValueError) as e:\n            raise EmbeddingAPIError(\n                f\"Embedding API at {self._url} returned malformed embeddings: {e}\"\n            ) from e\n        if arr.ndim != 2:\n            raise EmbeddingAPIError(\n                f\"Embedding API at {self._url} returned non-vector embeddings (shape {arr.shape})\"\n            )\n        # L2-normalize so cosine == dot product (collection uses\n        # hnsw:space=cosine), matching EmbeddinggemmaONNX above.\n        norms = np.linalg.norm(arr, axis=1, keepdims=True) + 1e-12\n        return (arr / norms).tolist()\n","sourceCodeStart":586,"sourceCodeEnd":622,"githubUrl":"https://github.com/MemPalace/mempalace/blob/06cb6987f02610784fefbad4b2bd5d026d164ba6/mempalace/embedding.py#L586-L622","documentation":"Raised by _vectors_from_response when, after sorting rows by 'index', the resulting indices are not exactly 0..n-1. The check exists because servers may return rows out of order (sorted first) but must not use absolute offsets (e.g. 5,6,7 for a second batch), duplicates, or gaps — positional alignment between texts and vectors would silently misassign embeddings, violating the 100%-recall promise. Alignment must be provably correct, so any deviation is fatal.","triggerScenarios":"A server that echoes the global input index instead of per-request 0-based index (common when a proxy batches multiple clients); duplicate index values from a server bug; missing rows combined with extras (e.g. indices [0,0,2] for n=3); a stub that omits 'index' entirely so all rows default to -1 after sorting.","commonSituations":"Aggregating gateways that re-index across concatenated batches; stub servers omitting the index field; homegrown servers copying absolute IDs from their internal queue into 'index'.","solutions":["Curl with a 3-text batch and verify each row has index 0, 1, 2 exactly (0-based, per request)","Fix stubs to include \"index\": i for every row","If a gateway re-batches, point mempalace directly at the model server or fix the gateway to renumber indices per request","Confirm no rows are dropped mid-pipeline (dropped rows create gaps that fail this check)"],"exampleFix":"# gateway pseudo-code — before\nresp['data'] = all_rows  # absolute indices from merged batch\n# after\nresp['data'] = [\n    {**row, 'index': i}\n    for i, row in enumerate(sorted(all_rows, key=lambda r: r['orig_pos']))\n]","handlingStrategy":"validation","validationCode":"resp = probe(url, model, inputs=[\"a\", \"b\"])\nidx = sorted(r.get(\"index\") for r in resp[\"data\"])\nassert idx == list(range(len(resp[\"data\"]))), f\"indices {idx} not 0-based contiguous\"","typeGuard":"def indices_are_contiguous(data) -> bool:\n    rows = data.get(\"data\", [])\n    got = sorted(r.get(\"index\", -1) for r in rows if isinstance(r, dict))\n    return got == list(range(len(rows)))","tryCatchPattern":"try:\n    vecs = ef(texts)\nexcept EmbeddingAPIError as e:\n    if \"non-contiguous\" in str(e):\n        log.error(\"server uses absolute/duplicate indices; fix gateway re-indexing\")\n    raise","preventionTips":["Bypass aggregating proxies for the embeddings route, or make them renumber 0..n-1 per request","Include an index-alignment assertion in your server's integration test"],"tags":["embedding","api","validation","index-alignment"],"backgroundTag":null,"analyzedSha":"06cb6987f02610784fefbad4b2bd5d026d164ba6","analyzedAt":"2026-08-15T03:03:36.213Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}