{"record":{"id":"e24165a8431e7124","repo":"vercel/turborepo","slug":"crlf-normalization-length-mismatch-scan-predicted","errorCode":null,"errorMessage":"CRLF normalization length mismatch: scan predicted {normalized_len}, stream produced {bytes_hashed}","messagePattern":"CRLF normalization length mismatch: scan predicted (.+?), stream produced (.+?)","errorType":"exception","errorClass":"std::io::Error","httpStatus":null,"severity":"error","filePath":"crates/turborepo-scm/src/crlf.rs","lineNumber":409,"sourceCode":"        };\n        return Ok((raw_hasher.finalize()?, outcome));\n    }\n\n    // CRLF normalization required — second pass with the normalized length.\n    // Safety: the file may have changed between the scan pass and this\n    // normalization pass (TOCTOU). The length check below detects this.\n    file.seek(SeekFrom::Start(0))?;\n    let normalized_len = scan.byte_count.saturating_sub(scan.crlf_count);\n\n    let mut hasher = BlobHasher::new();\n    hasher.write_blob_header(normalized_len);\n    let mut bytes_hashed: u64 = 0;\n    stream_normalized(file, |data| {\n        bytes_hashed += data.len() as u64;\n        hasher.update(data);\n    })?;\n    if bytes_hashed != normalized_len {\n        return Err(std::io::Error::new(\n            std::io::ErrorKind::InvalidData,\n            format!(\n                \"CRLF normalization length mismatch: scan predicted {normalized_len}, stream \\\n                 produced {bytes_hashed}\"\n            ),\n        ));\n    }\n    let outcome = HashOutcome {\n        normalized: true,\n        crlf_count: scan.crlf_count,\n    };\n    Ok((hasher.finalize()?, outcome))\n}\n\n/// Hash a working-tree file as a git blob (used when `.git/` is present).\n///\n/// Uses the `sha1` crate rather than gix's collision-detected SHA-1\n/// (`sha1_checked`, a Rust port of sha1dc). The `sha1` crate dispatches to","sourceCodeStart":391,"sourceCodeEnd":427,"githubUrl":"https://github.com/vercel/turborepo/blob/7fe373bc270adb30db3c184c55069f77feb1a6d6/crates/turborepo-scm/src/crlf.rs#L391-L427","documentation":"The fused scan+hash design hashes CRLF-normalized content in two passes: a first scan counts bytes and CRLFs, then the code seeks back to 0, computes normalized_len = byte_count - crlf_count, writes that into the git blob header, and streams the file again while hashing (crlf.rs:409 region). If the streamed byte count differs from the prediction, the file changed between the passes (TOCTOU); returning InvalidData 'CRLF normalization length mismatch' is what stops turbo from caching a wrong hash for mutated content.","triggerScenarios":"A file is written concurrently while turbo hashes it: a dev server/formatter rewriting the file, a build artifact being produced in a directory that is also an input, editor autosave racing `turbo run`.","commonSituations":"Hashing directories that another process actively writes into Running turbo while watch-mode tooling rewrites the same files Cache-warming scripts that mutate files as they enumerate them","solutions":["Re-run turbo — a one-off race usually resolves on the next attempt","Stop or pause concurrent writers (watchers, formatters) while hashing runs","Exclude the volatile files/dirs from task `inputs`/`outputs` so they are never scanned","If it persists with no writers, suspect a flaky disk/FUSE mount and check dmesg"],"exampleFix":"# before: hashing a dir that a watcher rewrites\nnpx turbo run build\n# after: exclude volatile dir from inputs in turbo.json\n\"inputs\": [\"$TURBO_DEFAULT$\", \"!live-data/**\"]","handlingStrategy":"retry","validationCode":"// detect concurrent writers before hashing: require a stable size/mtime across a short window\nlet a = std::fs::metadata(p)?; std::thread::sleep(Duration::from_millis(50));\nlet b = std::fs::metadata(p)?;\nif a.len() != b.len() || a.modified()? != b.modified()? { anyhow::bail!(\"file unstable\"); }","typeGuard":null,"tryCatchPattern":"// length mismatch == TOCTOU: re-run the hash from scratch\nfor attempt in 0..3 {\n    match hash_file_crlf(path) {\n        Err(e) if e.kind() == std::io::ErrorKind::InvalidData\n            && e.to_string().contains(\"length mismatch\") => continue,\n        r => break r?,\n    }\n}","preventionTips":["Pause watch-mode writers/formatters while hashing or caching","Exclude actively-rewritten files from task inputs/outputs","Never point turbo inputs at a directory another process produces into"],"tags":["cache","hashing","crlf","toctou","concurrency"],"backgroundTag":"file-changed-during-read","analyzedSha":"7fe373bc270adb30db3c184c55069f77feb1a6d6","analyzedAt":"2026-08-17T10:46:15.696Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}