overleaf/overleaf · error · OError

blocking doc insert into redis: doc is too large

Error message

blocking doc insert into redis: doc is too large

What it means

putDocInMemory throws this OError when inserting a doc into Redis whose size (per docIsTooLarge against Settings.max_doc_length) exceeds the allowed maximum. This only applies when shareJSTextOT is true, since editor-core's TextOperation enforces its own size check otherwise. It blocks oversized docs from entering Redis to prevent memory blowups.

Source

Thrown at services/document-updater/app/js/RedisManager.js:59

  ) {
    const timer = new metrics.Timer('redis.put-doc')
    const shareJSTextOT = Array.isArray(docLines)
    const docLinesArray = docLines
    docLines = JSON.stringify(docLines)
    if (docLines.indexOf('\u0000') !== -1) {
      // this check was added to catch memory corruption in JSON.stringify.
      // It sometimes returned null bytes at the end of the string.
      throw new OError('null bytes found in doc lines', { docId })
    }
    // Do an optimised size check on the docLines using the serialised
    // length as an upper bound
    const sizeBound = docLines.length
    if (
      shareJSTextOT && // editor-core has a size check in TextOperation.apply and TextOperation.applyToLength.
      docIsTooLarge(sizeBound, docLinesArray, Settings.max_doc_length)
    ) {
      const docSize = docLines.length
      throw new OError('blocking doc insert into redis: doc is too large', {
        projectId,
        docId,
        docSize,
      })
    }
    const docHash = RedisManager._computeHash(docLines)
    // record bytes sent to redis
    metrics.summary('redis.docLines', docLines.length, { status: 'set' })
    logger.debug(
      { projectId, docId, version, docHash, pathname, projectHistoryId },
      'putting doc in redis'
    )
    ranges = RedisManager._serializeRanges(ranges)

    // update docsInProject set before writing doc contents
    const projectBlockMulti = rclient.multi()
    projectBlockMulti.exists(keys.projectBlock({ project_id: projectId }))
    projectBlockMulti.sadd(keys.docsInProject({ project_id: projectId }), docId)

View on GitHub (pinned to 28ad3b03b7)

Solutions

1. Reduce the document content below Settings.max_doc_length before calling setDoc/putDocInMemory
2. Raise Settings.max_doc_length consistently across web/document-updater/spell-checker if larger docs are supported
3. Store very large content as a binary file in the project instead of a text doc
4. Check doc content for accidental duplication before writing

Example fix

// before
await redisManager.promises.putDocInMemory(projectId, docId, docLines, version, pathname, suppressHistoryReload)
// after
if (docLines.join('\n').length > Settings.max_doc_length) {
  throw new HttpError(413, 'doc too large')
}
await redisManager.promises.putDocInMemory(projectId, docId, docLines, version, pathname, suppressHistoryReload)

if (docLines.join('\n').length > Settings.max_doc_length) {
  throw new PayloadTooLargeError()
}

try {
  await putDocInMemory(...)
} catch (err) {
  if (err.message.includes('doc is too large')) return respond413()
  throw err
}

When it happens

Trigger: Calling RedisManager.putDocInMemory (via setDoc/flushDocIfOld/project load) where docIsTooLarge(sizeBound, docLinesArray, Settings.max_doc_length) is true and shareJSTextOT is enabled.

Common situations: Setting a doc's content to a very large payload via setDoc; loading a project whose docs exceed max_doc_length after a settings change; importing large text files into a doc rather than a binary file.

Related errors

• Project blocked from loading docs
• redis getDoc exceeded timeout
• redis getPreviousDocOps exceeded timeout
• Oops, something went wrong
• doc would become too large if appending this text

AI-assisted analysis of overleaf/overleaf@28ad3b03b7 (2026-09-03). Data as JSON: /api/errors/228dcf65584df416. Report an issue: GitHub.