{"record":{"id":"7ee2ca27a9a1acca","repo":"hcengineering/platform","slug":"unknown-object-size","errorCode":null,"errorMessage":"unknown object size","messagePattern":"unknown object size","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"foundations/server/packages/datalake/src/client.ts","lineNumber":238,"sourceCode":"    }\n  }\n\n  async putObject (\n    ctx: MeasureContext,\n    workspace: WorkspaceUuid,\n    objectName: string,\n    stream: Readable | Buffer | string,\n    params: UploadObjectParams\n  ): Promise<ObjectMetadata> {\n    let size = params.size\n    if (size === undefined) {\n      if (Buffer.isBuffer(stream)) {\n        size = stream.length\n      } else if (typeof stream === 'string') {\n        size = Buffer.byteLength(stream)\n      } else {\n        // TODO: Implement size calculation for Readable streams\n        ctx.warn('unknown object size', { workspace, objectName })\n      }\n    }\n\n    if (size === undefined || size < 64 * 1024 * 1024) {\n      return await ctx.with(\n        'direct-upload',\n        {},\n        (ctx) => this.uploadWithFormData(ctx, workspace, objectName, stream, { ...params, size }),\n        { workspace, objectName }\n      )\n    } else {\n      return await ctx.with(\n        'multipart-upload',\n        {},\n        (ctx) => this.uploadWithMultipart(ctx, workspace, objectName, stream, { ...params, size }),\n        { workspace, objectName }\n      )\n    }","sourceCodeStart":220,"sourceCodeEnd":256,"githubUrl":"https://github.com/hcengineering/platform/blob/63e28dc96483967b2fc21c881b3f1023c1de7718/foundations/server/packages/datalake/src/client.ts#L220-L256","documentation":"In DatalakeClient.putObject, the object size is only computed when the input is a Buffer (stream.length) or a string (Buffer.byteLength). For any other stream type (e.g. a Readable), size stays undefined and the client logs this warning, noting a TODO to implement size calculation for Readable streams. Upload then falls into a different (non-single-request) path since size is unknown.","triggerScenarios":"Calling putObject (directly or via put) with a Node Readable/other stream instead of a Buffer or string, so no known content length is available.","commonSituations":"Piping a file read stream, HTTP response body, or stream from another source directly into datalake put; refactored code that switched from readFile (Buffer) to createReadStream.","solutions":["Read the file fully into a Buffer before uploading (fs.promises.readFile) so the size is known","Compute and pass an explicit size option if the client API accepts one","For genuinely large/streamed data, accept the multipart path or buffer the stream yourself to determine length","Implement/guard the size calculation: track bytes as the Readable is consumed"],"exampleFix":"// before\nawait client.put(ctx, workspace, name, fs.createReadStream(path))\n// after\nconst buf = await fs.promises.readFile(path)\nawait client.put(ctx, workspace, name, buf)","handlingStrategy":"validation","validationCode":"function ensureSized(input: Buffer | string | Readable): Buffer | string {\n  if (Buffer.isBuffer(input) || typeof input === 'string') return input\n  throw new Error('datalake putObject requires a Buffer or string (known size)')\n}","typeGuard":"function hasKnownSize(s: unknown): s is Buffer | string {\n  return Buffer.isBuffer(s) || typeof s === 'string'\n}","tryCatchPattern":"try {\n  await client.put(ctx, ws, name, data)\n} catch (err) {\n  if (err instanceof Error && /unknown object size|size/i.test(err.message)) {\n    const buf = await streamToBuffer(data)\n    await client.put(ctx, ws, name, buf)\n  } else throw err\n}","preventionTips":["Always pass Buffers (readFile) rather than ReadStreams when object size is known","Build a streamToBuffer helper for stream inputs","Add a lint/test asserting uploads use sized inputs","Track byte counts manually if you must stream"],"tags":["datalake","streams","uploads"],"backgroundTag":"unknown-stream-size","analyzedSha":"63e28dc96483967b2fc21c881b3f1023c1de7718","analyzedAt":"2026-08-29T15:21:27.377Z","schemaVersion":2},"datasetVersion":"2026-08-29T17:17:51.833Z"}