vercel/turborepo · error · std::io::Error

CRLF normalization length mismatch: scan predicted {normaliz

Error message

CRLF normalization length mismatch: scan predicted {normalized_len}, stream produced {bytes_hashed}

What it means

The fused scan+hash design hashes CRLF-normalized content in two passes: a first scan counts bytes and CRLFs, then the code seeks back to 0, computes normalized_len = byte_count - crlf_count, writes that into the git blob header, and streams the file again while hashing (crlf.rs:409 region). If the streamed byte count differs from the prediction, the file changed between the passes (TOCTOU); returning InvalidData 'CRLF normalization length mismatch' is what stops turbo from caching a wrong hash for mutated content.

Source

Thrown at crates/turborepo-scm/src/crlf.rs:409

        };
        return Ok((raw_hasher.finalize()?, outcome));
    }

    // CRLF normalization required — second pass with the normalized length.
    // Safety: the file may have changed between the scan pass and this
    // normalization pass (TOCTOU). The length check below detects this.
    file.seek(SeekFrom::Start(0))?;
    let normalized_len = scan.byte_count.saturating_sub(scan.crlf_count);

    let mut hasher = BlobHasher::new();
    hasher.write_blob_header(normalized_len);
    let mut bytes_hashed: u64 = 0;
    stream_normalized(file, |data| {
        bytes_hashed += data.len() as u64;
        hasher.update(data);
    })?;
    if bytes_hashed != normalized_len {
        return Err(std::io::Error::new(
            std::io::ErrorKind::InvalidData,
            format!(
                "CRLF normalization length mismatch: scan predicted {normalized_len}, stream \
                 produced {bytes_hashed}"
            ),
        ));
    }
    let outcome = HashOutcome {
        normalized: true,
        crlf_count: scan.crlf_count,
    };
    Ok((hasher.finalize()?, outcome))
}

/// Hash a working-tree file as a git blob (used when `.git/` is present).
///
/// Uses the `sha1` crate rather than gix's collision-detected SHA-1
/// (`sha1_checked`, a Rust port of sha1dc). The `sha1` crate dispatches to

View on GitHub (pinned to 7fe373bc27)

Solutions

1. Re-run turbo — a one-off race usually resolves on the next attempt
2. Stop or pause concurrent writers (watchers, formatters) while hashing runs
3. Exclude the volatile files/dirs from task `inputs`/`outputs` so they are never scanned
4. If it persists with no writers, suspect a flaky disk/FUSE mount and check dmesg

Example fix

# before: hashing a dir that a watcher rewrites
npx turbo run build
# after: exclude volatile dir from inputs in turbo.json
"inputs": ["$TURBO_DEFAULT$", "!live-data/**"]

// detect concurrent writers before hashing: require a stable size/mtime across a short window
let a = std::fs::metadata(p)?; std::thread::sleep(Duration::from_millis(50));
let b = std::fs::metadata(p)?;
if a.len() != b.len() || a.modified()? != b.modified()? { anyhow::bail!("file unstable"); }

// length mismatch == TOCTOU: re-run the hash from scratch
for attempt in 0..3 {
    match hash_file_crlf(path) {
        Err(e) if e.kind() == std::io::ErrorKind::InvalidData
            && e.to_string().contains("length mismatch") => continue,
        r => break r?,
    }
}

When it happens

Trigger: A file is written concurrently while turbo hashes it: a dev server/formatter rewriting the file, a build artifact being produced in a directory that is also an input, editor autosave racing `turbo run`.

Common situations: Hashing directories that another process actively writes into Running turbo while watch-mode tooling rewrites the same files Cache-warming scripts that mutate files as they enumerate them

Related errors

• {path}: not a regular file
• failed to calculate Turbo hash: {err}
• path component contains NUL byte: {component:?}
• Windows returned invalid UTF-16 path: {e}
• error opening log file: {err:?}

AI-assisted analysis of vercel/turborepo@7fe373bc27 (2026-08-17). Data as JSON: /api/errors/e24165a8431e7124. Report an issue: GitHub.