tikv/tikv · error

currently only HDFS export is implemented

Error message

currently only HDFS export is implemented

What it means

The HDFS `ExternalStorage` implementation in components/external_storage supports only writing/exporting files to HDFS (via the `hdfs` CLI or libhdfs wrapper). Its `read` method is a deliberate `unimplemented!` stub: any attempt to read back a file from HDFS storage panics. Callers such as BR import/restore paths must not use HDFS storage as a readable source.

Source

Thrown at components/external_storage/src/hdfs.rs:142

            Ok(())
        } else {
            let stdout = String::from_utf8_lossy(&output.stdout);
            let stderr = String::from_utf8_lossy(&output.stderr);
            error!(
                "hdfs returned non-zero status";
                "code" => output.status.code(),
                "stdout" => stdout.as_ref(),
                "stderr" => stderr.as_ref(),
            );
            Err(io::Error::other(format!(
                "hdfs returned non-zero status: {:?}",
                output.status.code()
            )))
        }
    }

    fn read(&self, _name: &str) -> ExternalData<'_> {
        unimplemented!("currently only HDFS export is implemented")
    }

    fn read_part(&self, _name: &str, _off: u64, _len: u64) -> ExternalData<'_> {
        unimplemented!("currently only HDFS export is implemented")
    }

    /// Walk the prefix of the blob storage.
    /// It returns the stream of items.
    fn iter_prefix(
        &self,
        _prefix: &str,
    ) -> LocalBoxStream<'_, std::result::Result<BlobObject, io::Error>> {
        Box::pin(futures::future::err(crate::unimplemented()).into_stream())
    }

    fn delete(&self, _name: &str) -> LocalBoxFuture<'_, io::Result<()>> {
        Box::pin(futures::future::err(crate::unimplemented()))
    }

View on GitHub (pinned to 78aedc1c81)

Solutions

1. Read the data back via HDFS tooling outside the storage abstraction (e.g. `hdfs dfs -get`) and re-register it on a supported local/S3 storage.
2. Migrate the backup to a readable backend (S3/GCS/Azure/local) that implements `read`/`read_part`.
3. Patch `HdfsStorage::read` to shell out to `hdfs dfs -cat` (mirroring the existing write implementation) if HDFS reads are required.
4. Reconfigure the job so HDFS is only ever the export destination, never the read source.

Example fix

// before
let data = hdfs_storage.read("backup.sst")?; // panics
// after
run_hdfs_cmd(["dfs", "-get", &hdfs_url, &local_path])?; // mirror write() helper
let data = local_storage.read("backup.sst")?;

fn storage_supports_read(s: &dyn ExternalStorage) -> bool {
    // HdfsStorage only implements export (write)
    !s.url().scheme().eq_ignore_ascii_case("hdfs")
}
if !storage_supports_read(&storage) {
    eprintln!("HDFS storage cannot be read; stage data locally first");
}

fn is_hdfs_storage(url: &str) -> bool {
    url.starts_with("hdfs://")
}

match std::panic::catch_unwind(AssertUnwindSafe(|| storage.read(name))) {
    Ok(data) => data,
    Err(_) => {
        // fallback: fetch via hdfs CLI, then read locally
        run_hdfs_cmd(["dfs", "-get", &url, &local])?;
        local_storage.read(&local)?
    }
}

When it happens

Trigger: Calling `ExternalStorage::read("<name>")` on an `HdfsStorage` handle — e.g. BR/restore code or a tool configured with an `hdfs://` URL on the read side of a backup/restore or external-STS flow.

Common situations: Pointing a restore job at backup data left in `hdfs://` storage; using an HDFS storage config for external timestamp snapshotting where reads are required; testing HDFS storage locally.

Related errors

• panic!()
• panic!()
• only available for local mode
• not implemented
• SharedLocks returned from txn_types::parse_lock is not suppo

AI-assisted analysis of tikv/tikv@78aedc1c81 (2026-09-03). Data as JSON: /api/errors/c0783889151e64f6. Report an issue: GitHub.