{"record":{"id":"4d4026bd11bd178b","repo":"apache/hadoop","slug":"responsetimethresholds-must-match-with-the-number","errorCode":null,"errorMessage":"responseTimeThresholds must match with the number of priority levels","messagePattern":"responseTimeThresholds must match with the number of priority levels","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/DecayRpcScheduler.java","lineNumber":444,"sourceCode":"\n    for (int i = 0; i < ret.length; i++) {\n      ret[i] = Math.pow(2, i)/div;\n    }\n    return ret;\n  }\n\n  private static long[] parseBackOffResponseTimeThreshold(String ns,\n      Configuration conf, int numLevels) {\n    long[] responseTimeThresholds = conf.getTimeDurations(ns + \".\" +\n            IPC_DECAYSCHEDULER_BACKOFF_RESPONSETIME_THRESHOLDS_KEY,\n        TimeUnit.MILLISECONDS);\n    // backoff thresholds not specified\n    if (responseTimeThresholds.length == 0) {\n      return getDefaultBackOffResponseTimeThresholds(numLevels);\n    }\n    // backoff thresholds specified but not match with the levels\n    if (responseTimeThresholds.length != numLevels) {\n      throw new IllegalArgumentException(\n          \"responseTimeThresholds must match with the number of priority \" +\n          \"levels\");\n    }\n    // invalid thresholds\n    for (long responseTimeThreshold: responseTimeThresholds) {\n      if (responseTimeThreshold <= 0) {\n        throw new IllegalArgumentException(\n            \"responseTimeThreshold millis must be >= 0\");\n      }\n    }\n    return responseTimeThresholds;\n  }\n\n  // 10s for level 0, 20s for level 1, 30s for level 2, ...\n  private static long[] getDefaultBackOffResponseTimeThresholds(int numLevels) {\n    long[] ret = new long[numLevels];\n    for (int i = 0; i < ret.length; i++) {\n      ret[i] = 10000*(i+1);","sourceCodeStart":426,"sourceCodeEnd":462,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/DecayRpcScheduler.java#L426-L462","documentation":"parseBackOffResponseTimeThreshold reads <ns>.decay-scheduler.backoff.responsetime.thresholds (via Configuration.getTimeDurations, milliseconds). When response-time backoff is configured, the server needs one average-response-time threshold per priority level to decide when each queue starts rejecting calls; if the array length differs from numLevels it throws IllegalArgumentException at scheduler construction. Empty config is fine (defaults of 10s, 20s, 30s... per level).","triggerScenarios":"Enabling decay-scheduler.backoff.responsetime.enable=true and providing fewer/more thresholds than ipc.<port>.callqueue-priority-levels; changing the level count without updating the thresholds list; mixing duration units ('10s','20s','5s' with 4 levels).","commonSituations":"Rolling out response-time-based callqueue backoff on a busy NameNode; reusing a 3-level threshold list after raising priority levels to 4; HDFSRouterFed/YARN configs copied between services with different level counts.","solutions":["Set exactly numLevels comma-separated durations, e.g., 4 levels -> '3s,10s,20s,30s'.","Or drop the property and rely on the defaults (10s * (level+1)).","Keep the count in sync whenever callqueue-priority-levels changes."],"exampleFix":"// before (core-site.xml)\n<property><name>ipc.8020.decay-scheduler.backoff.responsetime.enable</name><value>true</value></property>\n<property><name>ipc.8020.decay-scheduler.backoff.responsetime.thresholds</name><value>10s,20s</value></property>\n<property><name>ipc.8020.callqueue-priority-levels</name><value>4</value></property>\n<!-- 2 thresholds for 4 levels -> IllegalArgumentException -->\n\n// after\n<property><name>ipc.8020.decay-scheduler.backoff.responsetime.thresholds</name><value>3s,10s,20s,30s</value></property>","handlingStrategy":"validation","validationCode":"int levels = conf.getInt(\"ipc.\" + port + \".callqueue-priority-levels\", 1);\nlong[] th = conf.getTimeDurations(\"ipc.\" + port + \".decay-scheduler.backoff.responsetime.thresholds\", TimeUnit.MILLISECONDS);\nif (th.length > 0 && th.length != levels) {\n  throw new ConfigException(\"backoff thresholds need exactly \" + levels + \" entries, got \" + th.length);\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["One backoff threshold per priority level, ascending durations are typical.","Re-validate whenever priority levels change during tuning.","Use explicit units ('10s') so counts and magnitudes stay readable."],"tags":["hadoop","ipc","decay-scheduler","backoff","thresholds","config"],"backgroundTag":"config-validation-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}