{"record":{"id":"c0783889151e64f6","repo":"tikv/tikv","slug":"currently-only-hdfs-export-is-implemented","errorCode":null,"errorMessage":"currently only HDFS export is implemented","messagePattern":"currently only HDFS export is implemented","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"components/external_storage/src/hdfs.rs","lineNumber":142,"sourceCode":"            Ok(())\n        } else {\n            let stdout = String::from_utf8_lossy(&output.stdout);\n            let stderr = String::from_utf8_lossy(&output.stderr);\n            error!(\n                \"hdfs returned non-zero status\";\n                \"code\" => output.status.code(),\n                \"stdout\" => stdout.as_ref(),\n                \"stderr\" => stderr.as_ref(),\n            );\n            Err(io::Error::other(format!(\n                \"hdfs returned non-zero status: {:?}\",\n                output.status.code()\n            )))\n        }\n    }\n\n    fn read(&self, _name: &str) -> ExternalData<'_> {\n        unimplemented!(\"currently only HDFS export is implemented\")\n    }\n\n    fn read_part(&self, _name: &str, _off: u64, _len: u64) -> ExternalData<'_> {\n        unimplemented!(\"currently only HDFS export is implemented\")\n    }\n\n    /// Walk the prefix of the blob storage.\n    /// It returns the stream of items.\n    fn iter_prefix(\n        &self,\n        _prefix: &str,\n    ) -> LocalBoxStream<'_, std::result::Result<BlobObject, io::Error>> {\n        Box::pin(futures::future::err(crate::unimplemented()).into_stream())\n    }\n\n    fn delete(&self, _name: &str) -> LocalBoxFuture<'_, io::Result<()>> {\n        Box::pin(futures::future::err(crate::unimplemented()))\n    }","sourceCodeStart":124,"sourceCodeEnd":160,"githubUrl":"https://github.com/tikv/tikv/blob/78aedc1c81ef3f7d8bacc6e9d09f56460f134937/components/external_storage/src/hdfs.rs#L124-L160","documentation":"The HDFS `ExternalStorage` implementation in components/external_storage supports only writing/exporting files to HDFS (via the `hdfs` CLI or libhdfs wrapper). Its `read` method is a deliberate `unimplemented!` stub: any attempt to read back a file from HDFS storage panics. Callers such as BR import/restore paths must not use HDFS storage as a readable source.","triggerScenarios":"Calling `ExternalStorage::read(\"<name>\")` on an `HdfsStorage` handle — e.g. BR/restore code or a tool configured with an `hdfs://` URL on the read side of a backup/restore or external-STS flow.","commonSituations":"Pointing a restore job at backup data left in `hdfs://` storage; using an HDFS storage config for external timestamp snapshotting where reads are required; testing HDFS storage locally.","solutions":["Read the data back via HDFS tooling outside the storage abstraction (e.g. `hdfs dfs -get`) and re-register it on a supported local/S3 storage.","Migrate the backup to a readable backend (S3/GCS/Azure/local) that implements `read`/`read_part`.","Patch `HdfsStorage::read` to shell out to `hdfs dfs -cat` (mirroring the existing write implementation) if HDFS reads are required.","Reconfigure the job so HDFS is only ever the export destination, never the read source."],"exampleFix":"// before\nlet data = hdfs_storage.read(\"backup.sst\")?; // panics\n// after\nrun_hdfs_cmd([\"dfs\", \"-get\", &hdfs_url, &local_path])?; // mirror write() helper\nlet data = local_storage.read(\"backup.sst\")?;","handlingStrategy":"fallback","validationCode":"fn storage_supports_read(s: &dyn ExternalStorage) -> bool {\n    // HdfsStorage only implements export (write)\n    !s.url().scheme().eq_ignore_ascii_case(\"hdfs\")\n}\nif !storage_supports_read(&storage) {\n    eprintln!(\"HDFS storage cannot be read; stage data locally first\");\n}","typeGuard":"fn is_hdfs_storage(url: &str) -> bool {\n    url.starts_with(\"hdfs://\")\n}","tryCatchPattern":"match std::panic::catch_unwind(AssertUnwindSafe(|| storage.read(name))) {\n    Ok(data) => data,\n    Err(_) => {\n        // fallback: fetch via hdfs CLI, then read locally\n        run_hdfs_cmd([\"dfs\", \"-get\", &url, &local])?;\n        local_storage.read(&local)?\n    }\n}","preventionTips":["Treat HDFS as write-only in BR/backup configuration.","Keep backup data on S3/GCS/Azure/local for restore paths.","Add a startup check that rejects hdfs:// URLs on read-side jobs.","If HDFS reads are needed, implement read/read_part via the hdfs CLI like write does."],"tags":["hdfs","external-storage","panic","unimplemented","backup-restore"],"backgroundTag":"hdfs-storage-read-unimplemented","analyzedSha":"78aedc1c81ef3f7d8bacc6e9d09f56460f134937","analyzedAt":"2026-09-03T23:31:32.398Z","contentChangedAt":"2026-09-03T23:31:32.398Z","schemaVersion":2},"datasetVersion":"2026-09-11T07:07:21.782Z"}