{"record":{"id":"33853379b16e1937","repo":"Mintplex-Labs/anything-llm","slug":"failed-to-get-youtube-video-transcription-e-me","errorCode":null,"errorMessage":"Failed to get YouTube video transcription: ${e?.message}","messagePattern":"Failed to get YouTube video transcription: (.+?)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"collector/utils/extensions/YoutubeTranscript/YoutubeLoader/index.js","lineNumber":76,"sourceCode":"        (module) => module.fetchTranscript\n      );\n      const transcriptSegments = await fetchTranscript(this.#videoId, {\n        lang: this.#language,\n      });\n      if (!transcriptSegments || transcriptSegments.length === 0)\n        throw new Error(\"Transcription not found\");\n      transcript = this.#convertTranscriptSegmentsToText(transcriptSegments);\n      if (this.#addVideoInfo) {\n        const { Innertube } = require(\"youtubei.js\");\n        const youtube = await Innertube.create();\n        const info = (await youtube.getBasicInfo(this.#videoId)).basic_info;\n        metadata.description = info.short_description;\n        metadata.title = info.title;\n        metadata.view_count = info.view_count;\n        metadata.author = info.author;\n      }\n    } catch (e) {\n      throw new Error(\n        `Failed to get YouTube video transcription: ${e?.message}`\n      );\n    }\n    return [\n      {\n        pageContent: transcript,\n        metadata,\n      },\n    ];\n  }\n\n  #convertTranscriptSegmentsToText(transcriptSegments) {\n    return transcriptSegments\n      .map((segment) =>\n        typeof segment === \"string\" ? segment : segment.text || \"\"\n      )\n      .join(\" \")\n      .replace(/\\s+/g, \" \")","sourceCodeStart":58,"sourceCodeEnd":94,"githubUrl":"https://github.com/Mintplex-Labs/anything-llm/blob/3aec848f2885144aa8f1e53b9731a04310d5d558/collector/utils/extensions/YoutubeTranscript/YoutubeLoader/index.js#L58-L94","documentation":"Same check as the cached path, applied to freshly embedded vectors: each batch goes to client.upsert(namespace, { wait: true, ... }) and anything other than additionResult.status === 'completed' throws. Note the surrounding try/catch converts this into { vectorized: false, error } rather than a crash, so the symptom is a failed document embed in API responses/logs. Non-completed means Qdrant did not durably acknowledge the write.","triggerScenarios":"Qdrant under load returns 'yellow' or times out on WAL flush; a batch larger than server limits; client/server version mismatch yielding a status field the code doesn't recognize (undefined/'ok'); transient network drop to a remote Qdrant Cloud cluster.","commonSituations":"Large documents split into 500-vector batches saturating a small Qdrant node; mixing old JS client with new server; shared dev instances with bursty concurrent embeds; flaky egress to cloud Qdrant.","solutions":["Check qdrant logs for WAL/optimizer/timeout messages at the failure timestamp.","Retry the document embed — single-batch yellow acks usually complete on retry.","Match @qdrant/js-client-rest to the server version so the upsert acknowledgment shape is the one expected.","Reduce concurrency of embed jobs or lower per-batch chunk size if the node is saturated."],"exampleFix":"// before - any non-'completed' ack marks the document failed\nconst additionResult = await client.upsert(namespace, { wait: true, batch });\nif (additionResult?.status !== 'completed')\n  throw new Error('Error embedding into QDrant', additionResult);\n\n// after - accept both durable ack spellings and retry transient ones\nasync function upsertCompleted(client, namespace, batch, tries = 3) {\n  for (let i = 0; i < tries; i++) {\n    const r = await client.upsert(namespace, { wait: true, batch });\n    if (r?.status === 'completed' || r?.status === 'ok') return r;\n    await new Promise((res) => setTimeout(res, 1000 * (i + 1)));\n  }\n  throw new Error(`Qdrant upsert not acknowledged: ${JSON.stringify(r)}`);\n}","handlingStrategy":"try-catch","validationCode":"// addDocumentToNamespace already swallows this into { vectorized:false, error } —\n// check the return value instead of awaiting a throw\nconst result = await vectorDB.addDocumentToNamespace(namespace, payload);\nif (!result.vectorized) {\n  logger.warn(`Embed failed for ${payload.docId}: ${result.error}`);\n  scheduleRetry(payload.docId);\n}","typeGuard":"function isDurableUpsertAck(r) {\n  return r?.status === 'completed' || r?.status === 'ok';\n}","tryCatchPattern":"try {\n  const r = await client.upsert(namespace, { wait: true, batch });\n  if (!['completed', 'ok'].includes(r?.status)) throw new Error('ack not durable');\n} catch (e) {\n  if (/Error embedding into QDrant/.test(e.message)) {\n    // bounded retry for transient yellow/timeout acks\n    for (let i = 1; i <= 3; i++) {\n      const r = await client.upsert(namespace, { wait: true, batch });\n      if (r?.status === 'completed') return;\n      await new Promise((res) => setTimeout(res, 1000 * i));\n    }\n  }\n  throw e;\n}","preventionTips":["Always inspect the { vectorized, error } return of addDocumentToNamespace rather than assuming success.","Match @qdrant/js-client-rest to the Qdrant server version before bulk ingest.","Throttle concurrent embeds; Qdrant yellow acks correlate with write saturation.","Retry failed documents idempotently (same ids upserted again)."],"tags":["qdrant","upsert","write-acknowledgment","retry","data-ingestion"],"backgroundTag":"vector-upsert-failed","analyzedSha":"3aec848f2885144aa8f1e53b9731a04310d5d558","analyzedAt":"2026-08-18T10:02:21.017Z","contentChangedAt":"2026-08-18T10:02:21.017Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}