{"record":{"id":"228dcf65584df416","repo":"overleaf/overleaf","slug":"blocking-doc-insert-into-redis-doc-is-too-large","errorCode":null,"errorMessage":"blocking doc insert into redis: doc is too large","messagePattern":"blocking doc insert into redis: doc is too large","errorType":"validation","errorClass":"OError","httpStatus":null,"severity":"error","filePath":"services/document-updater/app/js/RedisManager.js","lineNumber":59,"sourceCode":"  ) {\n    const timer = new metrics.Timer('redis.put-doc')\n    const shareJSTextOT = Array.isArray(docLines)\n    const docLinesArray = docLines\n    docLines = JSON.stringify(docLines)\n    if (docLines.indexOf('\\u0000') !== -1) {\n      // this check was added to catch memory corruption in JSON.stringify.\n      // It sometimes returned null bytes at the end of the string.\n      throw new OError('null bytes found in doc lines', { docId })\n    }\n    // Do an optimised size check on the docLines using the serialised\n    // length as an upper bound\n    const sizeBound = docLines.length\n    if (\n      shareJSTextOT && // editor-core has a size check in TextOperation.apply and TextOperation.applyToLength.\n      docIsTooLarge(sizeBound, docLinesArray, Settings.max_doc_length)\n    ) {\n      const docSize = docLines.length\n      throw new OError('blocking doc insert into redis: doc is too large', {\n        projectId,\n        docId,\n        docSize,\n      })\n    }\n    const docHash = RedisManager._computeHash(docLines)\n    // record bytes sent to redis\n    metrics.summary('redis.docLines', docLines.length, { status: 'set' })\n    logger.debug(\n      { projectId, docId, version, docHash, pathname, projectHistoryId },\n      'putting doc in redis'\n    )\n    ranges = RedisManager._serializeRanges(ranges)\n\n    // update docsInProject set before writing doc contents\n    const projectBlockMulti = rclient.multi()\n    projectBlockMulti.exists(keys.projectBlock({ project_id: projectId }))\n    projectBlockMulti.sadd(keys.docsInProject({ project_id: projectId }), docId)","sourceCodeStart":41,"sourceCodeEnd":77,"githubUrl":"https://github.com/overleaf/overleaf/blob/28ad3b03b71cb4311decdcb55c36b33ec10d72db/services/document-updater/app/js/RedisManager.js#L41-L77","documentation":"putDocInMemory throws this OError when inserting a doc into Redis whose size (per docIsTooLarge against Settings.max_doc_length) exceeds the allowed maximum. This only applies when shareJSTextOT is true, since editor-core's TextOperation enforces its own size check otherwise. It blocks oversized docs from entering Redis to prevent memory blowups.","triggerScenarios":"Calling RedisManager.putDocInMemory (via setDoc/flushDocIfOld/project load) where docIsTooLarge(sizeBound, docLinesArray, Settings.max_doc_length) is true and shareJSTextOT is enabled.","commonSituations":"Setting a doc's content to a very large payload via setDoc; loading a project whose docs exceed max_doc_length after a settings change; importing large text files into a doc rather than a binary file.","solutions":["Reduce the document content below Settings.max_doc_length before calling setDoc/putDocInMemory","Raise Settings.max_doc_length consistently across web/document-updater/spell-checker if larger docs are supported","Store very large content as a binary file in the project instead of a text doc","Check doc content for accidental duplication before writing"],"exampleFix":"// before\nawait redisManager.promises.putDocInMemory(projectId, docId, docLines, version, pathname, suppressHistoryReload)\n// after\nif (docLines.join('\\n').length > Settings.max_doc_length) {\n  throw new HttpError(413, 'doc too large')\n}\nawait redisManager.promises.putDocInMemory(projectId, docId, docLines, version, pathname, suppressHistoryReload)","handlingStrategy":"validation","validationCode":"if (docLines.join('\\n').length > Settings.max_doc_length) {\n  throw new PayloadTooLargeError()\n}","typeGuard":null,"tryCatchPattern":"try {\n  await putDocInMemory(...)\n} catch (err) {\n  if (err.message.includes('doc is too large')) return respond413()\n  throw err\n}","preventionTips":["Validate doc size at the API boundary before writing to Redis","Keep max_doc_length aligned across all services","Route oversized content to file storage instead of docs","Monitor doc sizes and alert before the limit"],"tags":["document-updater","doc-size-limit","redis"],"backgroundTag":"doc-too-large","analyzedSha":"28ad3b03b71cb4311decdcb55c36b33ec10d72db","analyzedAt":"2026-09-03T02:10:22.807Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-10T07:17:11.731Z"}