{"record":{"id":"133bc9ecbe309b39","repo":"nautechsystems/nautilus_trader","slug":"file-has-gz-extension-but-invalid-gzip-heade","errorCode":null,"errorMessage":"File '{}' has .gz extension but invalid gzip header","messagePattern":"File '(.+?)' has \\.gz extension but invalid gzip header","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/adapters/tardis/src/csv/mod.rs","lineNumber":146,"sourceCode":"            Ok(()) => break,\n            Err(e) => {\n                if attempt == MAX_RETRIES {\n                    anyhow::bail!(\n                        \"Failed to read gzip header from '{}' after {MAX_RETRIES} attempts: {e}\",\n                        filepath_ref.display()\n                    );\n                }\n                log::warn!(\n                    \"Attempt {attempt}/{MAX_RETRIES} failed to read header from '{}': {e}. Retrying after {DELAY_MS}ms...\",\n                    filepath_ref.display()\n                );\n                std::thread::sleep(Duration::from_millis(DELAY_MS));\n            }\n        }\n    }\n\n    if header_buf[0] != 0x1f || header_buf[1] != 0x8b {\n        anyhow::bail!(\n            \"File '{}' has .gz extension but invalid gzip header\",\n            filepath_ref.display()\n        );\n    }\n\n    for attempt in 1..=MAX_RETRIES {\n        match file.seek(SeekFrom::Start(0)) {\n            Ok(_) => break,\n            Err(e) => {\n                if attempt == MAX_RETRIES {\n                    anyhow::bail!(\n                        \"Failed to reset file position for '{}' after {MAX_RETRIES} attempts: {e}\",\n                        filepath_ref.display()\n                    );\n                }\n                log::warn!(\n                    \"Attempt {attempt}/{MAX_RETRIES} failed to seek in '{}': {e}. Retrying after {DELAY_MS}ms...\",\n                    filepath_ref.display()","sourceCodeStart":128,"sourceCodeEnd":164,"githubUrl":"https://github.com/nautechsystems/nautilus_trader/blob/18893faf8b356be3320add8de2f861b0b647cf06/crates/adapters/tardis/src/csv/mod.rs#L128-L164","documentation":"create_csv_reader validates that files with a .gz extension actually begin with the gzip magic bytes (0x1f 0x8b) before handing the reader to CSV parsing. The library throws this when the extension claims gzip compression but the file content is not a valid gzip stream, which would otherwise cause cryptic decompression failures downstream.","triggerScenarios":"Calling any Tardis CSV loader (load_trades, load_quotes, load_deltas, load_depth10_from_snapshot5, load_depth10_from_snapshot25, convert_options_chain_csv) with a filepath ending in .gz whose first two bytes are not 0x1f 0x8b — e.g. an uncompressed file renamed to .gz, or a truncated/HTML error page saved as .gz.","commonSituations":"A Tardis download failed and returned an HTML/JSON error page saved with the requested .gz name; a file was downloaded over plain HTTP without compression; a user gunzipped a file but kept the .gz extension.","solutions":["Verify the file is genuinely gzipped: run `file data.csv.gz` or `gunzip -t data.csv.gz`; re-download from Tardis if corrupt","If the file is actually uncompressed, rename it to .csv (remove the .gz extension)","Re-download the archive with a proper HTTP client, checking the response is a gzip stream","Confirm no proxy/CDN stripped the Content-Encoding during download"],"exampleFix":"// before (misnamed uncompressed file)\nloader.load_trades(\"trades.csv.gz\")?;\n// after\ngunzip -t trades.csv.gz  # verify first; if uncompressed:\n// loader.load_trades(\"trades.csv\")?;  (or `mv trades.csv.gz trades.csv`)","handlingStrategy":"validation","validationCode":"fn is_gzip(path: &Path) -> anyhow::Result<bool> {\n    use std::io::Read;\n    let mut f = std::fs::File::open(path)?;\n    let mut b = [0u8; 2];\n    f.read_exact(&mut b)?;\n    Ok(b == [0x1f, 0x8b])\n}\n// before loading:\nassert!(is_gzip(&path)?, \"{} is not a valid gzip file\", path.display());","typeGuard":null,"tryCatchPattern":"match loader.load_trades(&path) {\n    Ok(data) => data,\n    Err(e) if e.to_string().contains(\"invalid gzip header\") => {\n        eprintln!(\"File is not gzip; re-download or rename to .csv\");\n        return Err(e);\n    }\n    Err(e) => return Err(e),\n}","preventionTips":["Verify downloads with `gunzip -t` before ingesting","Never rename uncompressed files to .gz","Check Content-Encoding headers when downloading via HTTP clients"],"tags":["csv","gzip","file-validation","tardis"],"backgroundTag":"file-read-failed","analyzedSha":"18893faf8b356be3320add8de2f861b0b647cf06","analyzedAt":"2026-09-08T20:49:34.690Z","contentChangedAt":"2026-09-08T20:49:34.690Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}