{"record":{"id":"1b9fdf4fd0d50536","repo":"juicedata/juicefs","slug":"worker-config-is-too-large","errorCode":null,"errorMessage":"worker config is too large","messagePattern":"worker config is too large","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/sync/cluster.go","lineNumber":417,"sourceCode":"\t}\n\n\targs := []string{host, path}\n\targs = append(args, workerArgs...)\n\targs = append(args, \"--manager\", address)\n\tif !config.Verbose && !config.Quiet {\n\t\targs = append(args, \"-q\")\n\t}\n\treturn shellescape.EscapeArgs(args), payload, nil\n}\n\n// ReadClusterWorkerConfig reads storage URLs and environment variables from worker stdin.\nfunc ReadClusterWorkerConfig(r io.Reader) (string, string, map[string]string, error) {\n\tdata, err := io.ReadAll(io.LimitReader(r, maxClusterWorkerConfigSize+1))\n\tif err != nil {\n\t\treturn \"\", \"\", nil, fmt.Errorf(\"read worker config: %s\", err)\n\t}\n\tif len(data) > maxClusterWorkerConfigSize {\n\t\treturn \"\", \"\", nil, fmt.Errorf(\"worker config is too large\")\n\t}\n\tvar config clusterWorkerConfig\n\tif err := json.Unmarshal(data, &config); err != nil {\n\t\treturn \"\", \"\", nil, fmt.Errorf(\"unmarshal worker config: %s\", err)\n\t}\n\tif config.Source == \"\" || config.Destination == \"\" {\n\t\treturn \"\", \"\", nil, fmt.Errorf(\"worker config is missing source or destination\")\n\t}\n\treturn config.Source, config.Destination, config.Env, nil\n}\n\nfunc launchWorker(address string, config *Config, wg *sync.WaitGroup) {\n\tworkers := strings.Split(strings.Join(config.Workers, \",\"), \",\")\n\tfor _, host := range workers {\n\t\twg.Add(1)\n\t\tgo func(host string) {\n\t\t\tdefer wg.Done()\n\t\t\t// copy","sourceCodeStart":399,"sourceCodeEnd":435,"githubUrl":"https://github.com/juicedata/juicefs/blob/c9a67b23e8e08ec23ec331aa6f1675e2319e921c/pkg/sync/cluster.go#L399-L435","documentation":"ReadClusterWorkerConfig caps the stdin payload at maxClusterWorkerConfigSize bytes; it reads one extra byte to detect overflow. If the payload exceeds the cap it throws 'worker config is too large' instead of unmarshaling an oversized/possibly malicious config.","triggerScenarios":"The marshaled clusterWorkerConfig sent by the manager over stdin exceeds maxClusterWorkerConfigSize — typically because Env contains a very large number of variables or very long source/destination URLs.","commonSituations":"Sync launched from an environment with a huge inherited environment (hundreds of KBs of env vars, container with large injected config); extremely long presigned URLs with tokens; older manager version without size trimming paired with a newer worker.","solutions":["Reduce the environment size passed to the sync (unset unneeded variables before launching, or set --env-filter if available)","Shorten source/destination URLs (avoid embedding huge credentials/tokens in the URL)","Check maxClusterWorkerConfigSize in pkg/sync/cluster.go and, if legitimately needed, increase it in a local build","Verify manager and worker binary versions are compatible"],"exampleFix":"// before (huge env inherited)\njuicefs sync src dst --cluster w1\n// after (trim env)\nenv -i HOME=$HOME PATH=$PATH juicefs sync src dst --cluster w1","handlingStrategy":"validation","validationCode":"const maxClusterWorkerConfigSize = 1 << 20 // mirror of the code constant\nif len(payloadJSON) > maxClusterWorkerConfigSize {\n\treturn fmt.Errorf(\"worker env/URLs exceed %d bytes; trim environment before launching sync\", maxClusterWorkerConfigSize)\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Launch sync with a minimal environment (env -i or explicit whitelist)","Avoid embedding giant tokens in source/destination URLs","Keep manager and worker versions aligned so size limits match"],"tags":["sync","cluster","payload-limit","stdin"],"backgroundTag":"payload-too-large","analyzedSha":"c9a67b23e8e08ec23ec331aa6f1675e2319e921c","analyzedAt":"2026-09-06T17:55:48.476Z","contentChangedAt":"2026-09-06T17:55:48.476Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}