{"record":{"id":"37eb6041f24e4aa8","repo":"apache/dolphinscheduler","slug":"read-sql-content-error","errorCode":null,"errorMessage":"read sql content error","messagePattern":"read sql content error","errorType":"exception","errorClass":"TaskException","httpStatus":null,"severity":"error","filePath":"dolphinscheduler-task-plugin/dolphinscheduler-task-spark/src/main/java/org/apache/dolphinscheduler/plugin/task/spark/SparkTask.java","lineNumber":216,"sourceCode":"            String sqlContent = \"\";\n            String resourceFileName = \"\";\n            args.add(SparkConstants.SQL_FROM_FILE);\n            if (SparkConstants.TYPE_FILE.equals(sparkParameters.getSqlExecutionType())) {\n                final List<ResourceInfo> resourceInfos = sparkParameters.getResourceList();\n                if (resourceInfos.size() > 1) {\n                    log.warn(\"more than 1 files detected, use the first one by default\");\n                }\n\n                try {\n                    resourceFileName = resourceInfos.get(0).getResourceName();\n                    ResourceContext resourceContext = taskExecutionContext.getResourceContext();\n                    sqlContent = FileUtils.readFileToString(\n                            new File(\n                                    resourceContext.getResourceItem(resourceFileName).getResourceAbsolutePathInLocal()),\n                            StandardCharsets.UTF_8);\n                } catch (IOException e) {\n                    log.error(\"read sql content from file {} error \", resourceFileName, e);\n                    throw new TaskException(\"read sql content error\", e);\n                }\n            } else {\n                sqlContent = sparkParameters.getRawScript();\n            }\n            args.add(generateScriptFile(sqlContent));\n        }\n        return args;\n    }\n\n    private void populateSparkResourceDefinitions(List<String> args) {\n        int driverCores = sparkParameters.getDriverCores();\n        if (driverCores > 0) {\n            args.add(String.format(SparkConstants.DRIVER_CORES, driverCores));\n        }\n\n        String driverMemory = sparkParameters.getDriverMemory();\n        if (StringUtils.isNotEmpty(driverMemory)) {\n            args.add(String.format(SparkConstants.DRIVER_MEMORY, driverMemory));","sourceCodeStart":198,"sourceCodeEnd":234,"githubUrl":"https://github.com/apache/dolphinscheduler/blob/02eac45a1b6676e639fcbfb4be2243de5771b05d/dolphinscheduler-task-plugin/dolphinscheduler-task-spark/src/main/java/org/apache/dolphinscheduler/plugin/task/spark/SparkTask.java#L198-L234","documentation":"SparkTask.populateSparkOptions(), reached via getScript(), reads the user's SQL resource file from local storage with FileUtils.readFileToString(..., UTF_8) when the spark task is a SQL-type job running a stored script. An IOException during that read is rethrown as TaskException(\"read sql content error\"). The Spark job never starts; the failure is purely in loading the SQL source.","triggerScenarios":"The resource item returned by resourceContext.getResourceItem(resourceFileName) resolves to a local path that does not exist, was deleted by tenant-dir cleanup, or is unreadable due to permissions; the resource was moved/renamed in the resource center after the task definition referenced it.","commonSituations":"Resource center file deleted or renamed while the workflow still references its ID; worker runs as a user without read permission on the tenant resource directory; shared-storage (HDFS/S3) mount broken on the worker so the downloaded resource is absent locally.","solutions":["Verify the resource file still exists in the resource center and re-upload/re-select it if it was renamed or deleted, then re-save the task","Check the local resource path in the wrapped IOException and fix OS permissions so the worker OS user can read it","Confirm the worker's shared-storage mount (HDFS/S3 cache dir) is healthy; re-download the resource or restart the worker if the cache is stale"],"exampleFix":"// before: task references deleted resource\nresourceFileName = \"query.sql\"  // deleted from resource center\n// after: re-upload and re-select\n// Resource Center -> upload query.sql -> Spark SQL task -> Resource: query.sql -> save","handlingStrategy":"retry","validationCode":"File f = new File(resourceContext.getResourceItem(resourceFileName).getResourceAbsolutePathInLocal());\nif (!f.exists() || !f.canRead()) {\n    throw new IllegalStateException(\"SQL resource missing/unreadable: \" + f);\n}","typeGuard":"boolean resourceReadable(ResourceContext ctx, String name) {\n    try {\n        File f = new File(ctx.getResourceItem(name).getResourceAbsolutePathInLocal());\n        return f.isFile() && f.canRead();\n    } catch (Exception e) { return false; }\n}","tryCatchPattern":"try {\n    sparkTask.handle(callBack);\n} catch (TaskException e) {\n    // re-fetch/re-upload the resource, then retry once\n}","preventionTips":["Never delete or rename resources still referenced by workflows; check references first","Ensure the worker OS user has read access to tenant resource directories","Keep shared-storage mounts (HDFS/S3 cache) healthy on all workers"],"tags":["file-read","spark","resource"],"backgroundTag":"file-read-failed","analyzedSha":"02eac45a1b6676e639fcbfb4be2243de5771b05d","analyzedAt":"2026-09-06T17:43:00.555Z","contentChangedAt":"2026-09-06T17:43:00.555Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}