{"record":{"id":"caa3056258fcbb13","repo":"nautechsystems/nautilus_trader","slug":"options-chain-csv-rows-must-be-ordered-by-local-ti","errorCode":null,"errorMessage":"options_chain CSV rows must be ordered by local_timestamp when thinning","messagePattern":"options_chain CSV rows must be ordered by local_timestamp when thinning","errorType":"validation","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"crates/adapters/tardis/src/csv/convert.rs","lineNumber":129,"sourceCode":"            precision_by_instrument\n                .entry(instrument_id)\n                .or_insert_with(|| {\n                    OptionsChainPrecision::new(config.price_precision, config.size_precision)\n                })\n                .update(&record, config.price_precision, config.size_precision);\n            instrument_states\n                .entry(instrument_id)\n                .and_modify(|state| state.update_activation(record.local_timestamp))\n                .or_insert_with(|| InstrumentBuildState::new(record.clone()));\n\n            if let Some(interval) = config.snapshot_interval {\n                let interval_us = u64::try_from(interval.as_micros())\n                    .context(\"snapshot interval exceeds u64 microseconds\")?;\n                anyhow::ensure!(interval_us > 0, \"snapshot interval must be positive\");\n                let bucket = record.local_timestamp / interval_us;\n\n                if let Some(current_bucket) = current_bucket {\n                    anyhow::ensure!(\n                        bucket >= current_bucket,\n                        \"options_chain CSV rows must be ordered by local_timestamp when thinning\"\n                    );\n                }\n\n                if current_bucket.is_none_or(|current| bucket > current) {\n                    flush_pending_records_before(\n                        &catalog,\n                        &mut pending_records,\n                        &mut data_buffers,\n                        &precision_by_instrument,\n                        bucket,\n                        config.extract_bbo_as_quotes,\n                    )?;\n                    current_bucket = Some(bucket);\n                }\n                pending_records\n                    .entry((instrument_id, bucket))","sourceCodeStart":111,"sourceCodeEnd":147,"githubUrl":"https://github.com/nautechsystems/nautilus_trader/blob/18893faf8b356be3320add8de2f861b0b647cf06/crates/adapters/tardis/src/csv/convert.rs#L111-L147","documentation":"When snapshot thinning is enabled for options-chain CSV conversion, rows are bucketed by `local_timestamp / interval_us` and only the first row of each bucket is kept. This requires the CSV to be sorted ascending by `local_timestamp`. If a later row belongs to an earlier bucket than the current one, the input is out of order and thinning would produce incorrect snapshots, so the conversion aborts.","triggerScenarios":"Calling `convert_options_chain_csv` with a `snapshot_interval` set while the CSV contains rows whose `local_timestamp` decreases relative to previously processed rows (within or across files in a multi-file run).","commonSituations":"Concatenating Tardis CSV exports in the wrong order; a provider export that is sorted by a different timestamp column (e.g. `timestamp` instead of `local_timestamp`); appending new data to an existing file without re-sorting.","solutions":["Sort the CSV by `local_timestamp` ascending before conversion, e.g. `sort -t, -k<col> file.csv` or a pandas `sort_values(\"local_timestamp\")` pass.","Reorder the list of input files chronologically (the bucket state resets per file but ordering within each file still matters).","If the data cannot be ordered, disable thinning (`snapshot_interval: None`), which skips the ordering check.","Verify the export tool is sorting by `local_timestamp`, not `timestamp`."],"exampleFix":"// before: convert unsorted CSV directly\nconvert_options_chain_csv(&path, &config)?;\n// after: sort first\nlet mut df = CsvParser::from_path(&path)?;\ndf.sort([\"local_timestamp\"]);\ndf.to_csv(&path)?;\nconvert_options_chain_csv(&path, &config)?;","handlingStrategy":"validation","validationCode":"import csv\n\ndef ensure_sorted_by_local_timestamp(path: str, col: str = \"local_timestamp\") -> None:\n    prev = None\n    with open(path, newline=\"\") as f:\n        for row in csv.DictReader(f):\n            ts = int(row[col])\n            if prev is not None and ts < prev:\n                raise ValueError(f\"{path}: {col} out of order at ts={ts}\")\n            prev = ts","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always sort exports by local_timestamp before conversion (pandas sort_values or GNU sort)","Order multi-file inputs chronologically before feeding them to the converter","Confirm the exporter sorts by local_timestamp, not timestamp"],"tags":["rust","csv","data-ordering","validation"],"backgroundTag":"invalid-argument-format","analyzedSha":"18893faf8b356be3320add8de2f861b0b647cf06","analyzedAt":"2026-09-08T20:49:34.690Z","contentChangedAt":"2026-09-08T20:49:34.690Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}