{"record":{"id":"2bae8017e66cf20c","repo":"FuelLabs/fuel-core","slug":"couldn-t-create-environment-for-backup","errorCode":null,"errorMessage":"Couldn't create environment for backup: {}","messagePattern":"Couldn't create environment for backup: (.+?)","errorType":"exception","errorClass":"DatabaseError::BackupEngineInitError","httpStatus":null,"severity":"error","filePath":"crates/fuel-core/src/state/rocks_db.rs","lineNumber":795,"sourceCode":"        let backup_dir = backup_dir.as_ref().join(Description::name());\n        let backup_dir_path = backup_dir.as_path();\n\n        let mut backup_engine_options = BackupEngineOptions::new(backup_dir_path)\n            .map_err(|e| {\n                DatabaseError::BackupEngineInitError(anyhow::anyhow!(\n                    \"Couldn't create backup engine options for path `{}`: {}\",\n                    backup_dir_path.display(),\n                    e\n                ))\n            })?;\n\n        let cpu_number =\n            i32::try_from(num_cpus::get()).expect(\"The number of CPU can't exceed `i32`\");\n\n        backup_engine_options.set_max_background_operations(cmp::max(1, cpu_number / 4));\n\n        let env = Env::new().map_err(|e| {\n            DatabaseError::BackupEngineInitError(anyhow::anyhow!(\n                \"Couldn't create environment for backup: {}\",\n                e\n            ))\n        })?;\n\n        let backup_engine =\n            BackupEngine::open(&backup_engine_options, &env).map_err(|e| {\n                DatabaseError::BackupEngineInitError(anyhow::anyhow!(\n                    \"Couldn't open backup engine for path `{}`: {}\",\n                    backup_dir.display(),\n                    e\n                ))\n            })?;\n\n        Ok(backup_engine)\n    }\n\n    #[cfg(feature = \"backup\")]","sourceCodeStart":777,"sourceCodeEnd":813,"githubUrl":"https://github.com/FuelLabs/fuel-core/blob/b9d4d170da3a31c9ace5f963d633b326348e0d42/crates/fuel-core/src/state/rocks_db.rs#L777-L813","documentation":"Creating the RocksDB Env (environment and thread-pool handle) for the backup engine failed, wrapped as DatabaseError::BackupEngineInitError. Env::new is a thin wrapper over OS resource creation, so failures are rare and system-level: file-descriptor exhaustion, inability to spawn threads, or severe memory pressure.","triggerScenarios":"Running backup while the process is near OS limits — out of file descriptors (ulimit -n), thread creation blocked by a pids cgroup limit, or heavy memory pressure.","commonSituations":"Backups scheduled on heavily loaded nodes with many RocksDB background threads; restrictive container limits; small default ulimits under systemd service managers.","solutions":["Raise file-descriptor and process limits (ulimit -n, systemd LimitNOFILE, container pids/memory limits).","Reduce load or memory pressure before running the backup, including lowering rocksdb background thread counts.","Retry once pressure is relieved; if it persists, inspect the trailing error plus dmesg or OOM logs."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// Pre-flight resource sanity check (approximate)\nfn resources_sane() -> bool {\n    let fds = std::fs::read_to_string(\"/proc/self/limits\").map(|s| {\n        s.lines().any(|l| l.starts_with(\"Max open files\") && !l.contains(\"unlimited\"))\n    });\n    let threads = std::thread::available_parallelism().map(|n| n.get() > 0).unwrap_or(false);\n    fds.is_ok() && threads\n}","typeGuard":null,"tryCatchPattern":"match RocksDb::<Description>::backup(db_dir, backup_dir, db_config) {\n    Err(e) if e.to_string().contains(\"Couldn't create environment for backup\") => {\n        // raise ulimits / container limits, reduce load, then retry once\n    }\n    rest => rest,\n}","preventionTips":["Set generous LimitNOFILE and pids limits for nodes that run backups.","Schedule backups during low-load windows.","Monitor fd usage and memory before backup jobs."],"tags":["rocksdb","backup","system-resources","fuel-core","rust"],"backgroundTag":null,"analyzedSha":"b9d4d170da3a31c9ace5f963d633b326348e0d42","analyzedAt":"2026-08-16T08:56:42.692Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}