hcengineering/platform · error · DatalakeError

Failed to upload multipart part

Error message

Failed to upload multipart part

What it means

DatalakeError thrown by the datalake client when the PUT request that uploads a single part of a multipart upload fails. It wraps any lower-level error (NetworkError, NotFoundError, or non-OK HTTP response from fetchSafe) into one consistent error type; the underlying cause is logged via ctx.error with workspace/objectName. It means the chunk never reached the server or the server rejected the request.

Source

Thrown at foundations/server/packages/datalake/src/client.ts:433

    data: Readable | Buffer | string
  ): Promise<MultipartUploadPart> {
    const path = `/upload/multipart/${workspace}/${encodeURIComponent(objectName)}/part`
    const url = new URL(concatLink(this.endpoint, path))
    url.searchParams.set('uploadId', multipart.uploadId)
    url.searchParams.set('partNumber', partNumber.toString())

    const body = data instanceof Readable ? (Readable.toWeb(data) as ReadableStream) : data

    try {
      const response = await fetchSafe(ctx, url, {
        method: 'PUT',
        body: body as BodyInit,
        headers: { ...this.headers }
      })
      return (await response.json()) as MultipartUploadPart
    } catch (err: any) {
      ctx.error('failed to upload multipart part', { workspace, objectName, err })
      throw new DatalakeError('Failed to upload multipart part')
    }
  }

  private async multipartUploadComplete (
    ctx: MeasureContext,
    workspace: WorkspaceUuid,
    objectName: string,
    multipart: MultipartUpload,
    parts: MultipartUploadPart[]
  ): Promise<ObjectMetadata> {
    const path = `/upload/multipart/${workspace}/${encodeURIComponent(objectName)}/complete`
    const url = new URL(concatLink(this.endpoint, path))
    url.searchParams.set('uploadId', multipart.uploadId)

    try {
      const res = await fetchSafe(ctx, url, {
        method: 'POST',
        body: JSON.stringify({ parts }),

View on GitHub (pinned to 63e28dc964)

Solutions

1. Check the ctx.error log entry for the wrapped err to see the real cause.
2. Verify the datalake endpoint URL and auth headers (this.endpoint/this.headers) are correct.
3. Retry the whole uploadWithMultipart call - a fresh uploadId fixes stale-session failures.
4. Check network connectivity/proxies between client and datalake service.

Example fix

// before
await datalake.put(ctx, wsIds, name, hugeStream)
// after
try {
  await datalake.put(ctx, wsIds, name, hugeStream)
} catch (err) {
  if (err instanceof DatalakeError) {
    console.error('multipart part upload failed, retrying')
    await datalake.put(ctx, wsIds, name, hugeStream)
  } else throw err
}

const isUploadable = (d: unknown): d is Buffer | string | Readable =>
  Buffer.isBuffer(d) || typeof d === 'string' || d instanceof Readable
if (!isUploadable(data)) throw new TypeError('body must be Buffer, string, or Readable')

const isUploadable = (d: unknown): d is Buffer | string | Readable =>
  Buffer.isBuffer(d) || typeof d === 'string' || d instanceof Readable

try {
  await datalake.put(ctx, wsIds, name, data)
} catch (err) {
  if (err instanceof DatalakeError) {
    await sleep(backoff) // root cause is in ctx.error logs
    await datalake.put(ctx, wsIds, name, data)
  } else throw err
}

When it happens

Trigger: uploadWithMultipart (via put) with a body large enough to be chunked, where the PUT to /upload/multipart/{workspace}/{object}/part fails: fetch rejects (network down, DNS, TLS) or the server returns non-OK (401 bad token, 404 stale uploadId, 500 server error).

Common situations: Datalake service restarted mid-upload leaving a stale uploadId; wrong endpoint/auth headers in config; uploading from a stream that errors mid-read; transient network blips on large files.

Related errors

• Failed to complete multipart upload
• response.statusText
• Failed to fetch config
• unknownError(response.statusText)
• Failed to delete file

AI-assisted analysis of hcengineering/platform@63e28dc964 (2026-08-29). Data as JSON: /api/errors/5107671340e4e40e. Report an issue: GitHub.