{"record":{"id":"bee83576a7f4f68c","repo":"sinelaw/fresh","slug":"0-mb-requires-full-load-l-oad-e-ncoding-c-ancel","errorCode":null,"errorMessage":"{} ({:.0} MB) requires full load. (l)oad, (e)ncoding, (C)ancel? ","messagePattern":"(.+?) \\((.+?) MB\\) requires full load\\. \\(l\\)oad, \\(e\\)ncoding, \\(C\\)ancel\\? ","errorType":"error_code","errorClass":"LargeFileEncodingConfirmation","httpStatus":null,"severity":"warning","filePath":"crates/fresh-editor-core/src/model/buffer/mod.rs","lineNumber":643,"sourceCode":"            format::detect_encoding_or_binary(&sample, file_size > sample_size);\n        let is_binary = detected_binary && !force_text;\n\n        // Binary files skip encoding conversion to preserve raw bytes — and,\n        // like large text, are never slurped: reading a 2 GB zip whole cost\n        // ~1.2x its size resident and froze the editor thread for the read plus\n        // the line scan (issue #3142). A binary buffer opens read-only, so the\n        // lazy piece below serves it just as well.\n        //\n        // The `requires_full_load` gates below stop a non-resynchronizable\n        // *text* encoding being decoded chunk-wise; binary is never decoded, so\n        // they do not apply to it — as before, when it returned above them.\n        if !is_binary {\n            // Check if encoding requires full file loading\n            let requires_full_load = encoding.requires_full_file_load();\n\n            // For non-resynchronizable encodings, require confirmation unless forced\n            if requires_full_load && !force_full_load {\n                anyhow::bail!(LargeFileEncodingConfirmation {\n                    path: path.to_path_buf(),\n                    file_size,\n                    encoding,\n                });\n            }\n        }\n\n        // For encodings that require full load (non-resynchronizable or non-UTF-8),\n        // load the entire file and convert\n        if !is_binary && !matches!(encoding, Encoding::Utf8 | Encoding::Ascii) {\n            tracing::info!(\n                \"Large file with non-UTF-8 encoding ({:?}), loading fully for conversion\",\n                encoding\n            );\n            let contents = fs.read_file(path)?;\n            let mut buffer = Self::from_bytes_normalizing(contents, fs, normalize_cr);\n            buffer.persistence.set_file_path(path.to_path_buf());\n            buffer.persistence.clear_modified();","sourceCodeStart":625,"sourceCodeEnd":661,"githubUrl":"https://github.com/sinelaw/fresh/blob/67894ca5463dbd7a89bb31add4627c27d6b79d83/crates/fresh-editor-core/src/model/buffer/mod.rs#L625-L661","documentation":"When opening a large file, load_large_file_internal can defer full loading (partial/lazy loading) — but only for encodings that support it. If the chosen encoding requires reading the entire file (requires_full_file_load()) and the call wasn't forced, the loader bails out with LargeFileEncodingConfirmation so the UI can ask the user to confirm a full load, pick another encoding, or cancel.","triggerScenarios":"Loading a file that exceeds the large-file threshold with a non-binary encoding whose requires_full_file_load() returns true (e.g. encodings needing full-file conversion like UTF-16 with detection, or non-resynchronizable encodings), without setting force_full_load=true on the load call.","commonSituations":"Opening a multi-GB UTF-16 or non-UTF-8 encoded log/data file; users hitting the interactive prompt '(l)oad, (e)ncoding, (C)ancel?' and either choosing an option; scripts or tests calling the loader programmatically without force_full_load and being surprised by the bail.","solutions":["Handle the LargeFileEncodingConfirmation error in your caller: surface the (l)oad/(e)ncoding/(C)ancel choice to the user and retry accordingly.","Retry the load with force_full_load=true to proceed with a full read (mind memory usage for very large files).","Retry with a different encoding that does not require full-file loading (e.g. a resynchronizable/UTF-8-compatible one) to keep lazy loading.","Cancel if full load is unacceptable; the file is not loaded and no data is modified."],"exampleFix":"// before\nbuffer.load_large_file(path, file_size, encoding, /*force_full_load*/ false)?;\n\n// after: confirm or force based on user decision\nmatch buffer.load_large_file(path, file_size, encoding, false) {\n    Err(e) if e.downcast_ref::<LargeFileEncodingConfirmation>().is_some() => {\n        if user_chose_load {\n            buffer.load_large_file(path, file_size, encoding, true)?;\n        }\n    }\n    r => r?,\n}","handlingStrategy":"validation","validationCode":"// before loading a large file, check whether the encoding forces a full read\nlet needs_full = file_size > LARGE_FILE_THRESHOLD\n    && !is_binary\n    && encoding.requires_full_file_load();","typeGuard":null,"tryCatchPattern":"match buffer.load_large_file(path, size, enc, false) {\n    Err(e) if e.downcast_ref::<LargeFileEncodingConfirmation>().is_some() => {\n        // show (l)oad / (e)ncoding / (C)ancel prompt, then retry or abort\n    }\n    other => other?,\n}","preventionTips":["Check encoding.requires_full_file_load() before choosing lazy loading for big files.","Prefer resynchronizable/UTF-8-compatible encodings for large-file workflows.","Never pass force_full_load=true blindly on multi-GB files; confirm memory headroom first."],"tags":["large-file","encoding","user-confirmation","lazy-loading"],"backgroundTag":"confirmation-required","analyzedSha":"67894ca5463dbd7a89bb31add4627c27d6b79d83","analyzedAt":"2026-09-13T15:04:03.701Z","contentChangedAt":"2026-09-13T15:04:03.701Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}