{"record":{"id":"04c48d5fec7c2bb1","repo":"HKUDS/DeepTutor","slug":"ima-accepts-at-most-max-import-urls-urls-per-cal","errorCode":null,"errorMessage":"IMA accepts at most {MAX_IMPORT_URLS} URLs per call.","messagePattern":"IMA accepts at most (.+?) URLs per call\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"deeptutor/services/rag/pipelines/ima/client.py","lineNumber":224,"sourceCode":"        )\n\n    # ----- writing --------------------------------------------------------\n\n    async def import_urls(self, urls: list[str], *, folder_id: str = \"\") -> list[ImaImportedUrl]:\n        \"\"\"Add up to :data:`MAX_IMPORT_URLS` web pages to the bound library.\n\n        IMA reports a per-URL verdict rather than failing the batch, so partial\n        success is normal and is returned as-is for the caller to report.\n        \"\"\"\n        cleaned: list[str] = []\n        for raw in urls:\n            url = str(raw or \"\").strip()\n            if url and url not in cleaned:\n                cleaned.append(url)\n        if not cleaned:\n            raise ValueError(\"At least one URL is required.\")\n        if len(cleaned) > MAX_IMPORT_URLS:\n            raise ValueError(f\"IMA accepts at most {MAX_IMPORT_URLS} URLs per call.\")\n\n        # ``folder_id`` is required here, and the root folder's id is the\n        # knowledge base id itself.\n        target = str(folder_id or \"\").strip() or self._config.knowledge_base_id\n        data = await self._wire.post(\n            \"import_urls\",\n            {\n                \"urls\": cleaned,\n                \"knowledge_base_id\": self._config.knowledge_base_id,\n                \"folder_id\": target,\n            },\n        )\n        results = parse_imported_urls(data)\n        if results:\n            return results\n        # Some responses acknowledge the batch without echoing per-URL rows.\n        return [ImaImportedUrl(url=url, ok=True) for url in cleaned]\n","sourceCodeStart":206,"sourceCodeEnd":242,"githubUrl":"https://github.com/HKUDS/DeepTutor/blob/3e82f130422a813cdd73c10b21a44e9325f5821a/deeptutor/services/rag/pipelines/ima/client.py#L206-L242","documentation":"ValueError from IMA client import_urls: after cleaning/dedup, more than MAX_IMPORT_URLS URLs were supplied in a single call; the IMA endpoint enforces a per-request batch cap.","triggerScenarios":"Calling import_urls with a list longer than MAX_IMPORT_URLS (deduplicated count is what's checked, so removing duplicates alone won't help past the cap).","commonSituations":"Bulk-import scripts feeding dozens/hundreds of URLs at once; UI allowing arbitrary multi-paste without chunking.","solutions":["Chunk the URL list into batches of MAX_IMPORT_URLS and issue one import_urls call per chunk.","Add a client-side count check against the limit before submitting.","Trim unnecessary URLs from the batch."],"exampleFix":"# before\nawait client.import_urls(folder_id, all_500_urls)\n# after\nfor chunk in batched(urls, MAX_IMPORT_URLS):\n    await client.import_urls(folder_id, list(chunk))","handlingStrategy":"validation","validationCode":"cleaned = dedupe(urls)\nassert 0 < len(cleaned) <= MAX_IMPORT_URLS, \"chunk the URL list\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Chunk imports at MAX_IMPORT_URLS in every batch-import script."],"tags":["ima","rate-limit","batch-size","input-validation"],"backgroundTag":"request-batch-limit-exceeded","analyzedSha":"3e82f130422a813cdd73c10b21a44e9325f5821a","analyzedAt":"2026-08-27T06:57:25.364Z","schemaVersion":2},"datasetVersion":"2026-08-27T08:17:20.692Z"}