wavetermdev/waveterm · error

file data size %d exceeds transfer limit of %d bytes

Error message

file data size %d exceeds transfer limit of %d bytes

What it means

PutFile enforces RemoteFileTransferSizeLimit (32 MiB) on the decoded size of data.Data64, computed via base64.StdEncoding.DecodedLen. Writes whose payload exceeds this limit are rejected locally with this error instead of being sent to the remote, protecting against unbounded RPC payloads.

Source

Thrown at pkg/remote/fileshare/wshfs/wshfs.go:171

	if err != nil {
		return nil, err
	}
	return stat(conn)
}

func stat(conn *connparse.Connection) (*wshrpc.FileInfo, error) {
	return wshclient.RemoteFileInfoCommand(RpcClient, conn.Path, &wshrpc.RpcOpts{Route: wshutil.MakeConnectionRouteId(conn.Host)})
}

func PutFile(ctx context.Context, data wshrpc.FileData) error {
	log.Printf("PutFile: %v", data.Info.Path)
	conn, err := parseConnection(ctx, data.Info.Path)
	if err != nil {
		return err
	}
	dataSize := base64.StdEncoding.DecodedLen(len(data.Data64))
	if dataSize > RemoteFileTransferSizeLimit {
		return fmt.Errorf("file data size %d exceeds transfer limit of %d bytes", dataSize, RemoteFileTransferSizeLimit)
	}
	info := data.Info
	if info == nil {
		info = &wshrpc.FileInfo{Opts: &wshrpc.FileOpts{}}
	} else if info.Opts == nil {
		info.Opts = &wshrpc.FileOpts{}
	}
	info.Path = conn.Path
	info.Opts.Truncate = true
	data.Info = info
	return wshclient.RemoteWriteFileCommand(RpcClient, data, &wshrpc.RpcOpts{Route: wshutil.MakeConnectionRouteId(conn.Host)})
}

func Append(ctx context.Context, data wshrpc.FileData) error {
	log.Printf("Append: %v", data.Info.Path)
	conn, err := parseConnection(ctx, data.Info.Path)
	if err != nil {
		return err

View on GitHub (pinned to a4447c1563)

Solutions

1. Split the write: write the first chunk with PutFile then add remaining data with wshfs.Append in <=32 MiB chunks
2. Compress data before base64-encoding if it compresses well
3. Use the streaming APIs (FileStream/stream write) instead of a single PutFile for large files
4. Check file size with Stat before writing to decide on a chunked strategy

Example fix

// before
if len(content) > 40*1024*1024 {
    wshfs.PutFile(ctx, wshrpc.FileData{Info: info, Data64: b64(content)}) // errors
}
// after
const chunk = 8 * 1024 * 1024
first := true
for off := 0; off < len(content); off += chunk {
    end := min(off+chunk, len(content))
    b64data := base64.StdEncoding.EncodeToString(content[off:end])
    if first {
        err = wshfs.PutFile(ctx, wshrpc.FileData{Info: info, Data64: b64data}); first = false
    } else {
        err = wshfs.Append(ctx, wshrpc.FileData{Info: info, Data64: b64data})
    }
}

const transferLimit = 32 * 1024 * 1024
if base64.StdEncoding.DecodedLen(len(data.Data64)) > transferLimit {
    return fmt.Errorf("payload exceeds wshfs 32 MiB PutFile limit")
}

err := wshfs.PutFile(ctx, data)
if err != nil {
    if strings.Contains(err.Error(), "exceeds transfer limit") {
        return chunkedWrite(ctx, data) // PutFile first chunk + Append rest
    }
    return err
}

When it happens

Trigger: Calling wshfs.PutFile (via FileCreateCommand/FileWriteCommand) with Data64 whose base64 payload decodes to more than 32*1024*1024 bytes — e.g. base64.DecodedLen(len(Data64)) > 33554432.

Common situations: Writing a large binary, video, database dump, or log file in a single PutFile call; uploading a file that grew past the limit since the code was written; copying files without chunking.

Related errors

• error parsing destination connection: %w
• cannot copy %q to %q: %w
• move internal, src and dest hosts do not match
• beginning of file
• procinfo: process not found

AI-assisted analysis of wavetermdev/waveterm@a4447c1563 (2026-09-01). Data as JSON: /api/errors/88cff1f440ee6ba4. Report an issue: GitHub.