{"record":{"id":"e4df0d11430c4d15","repo":"apache/hadoop","slug":"fs-azure-blob-dir-list-consumer-max-lag","errorCode":"fs.azure.blob.dir.list.consumer.max.lag","errorMessage":"maxConsumptionLag should be lesser than maxSize","messagePattern":"maxConsumptionLag should be lesser than maxSize","errorType":"validation","errorClass":"InvalidConfigurationValueException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ListBlobQueue.java","lineNumber":86,"sourceCode":"   * it will wait for 5000 items to be consumed before generating next 5000 items.\n   * <p>\n   * If this is not used, the producer will keep on producing items as soon as\n   * the queue become available with small size. Let say, 5 items got consumed,\n   * producer would make a server call for only 5 items and would populate the queue.\n   * <p>\n   * This mechanism would prevent producer making server calls for very small amount\n   * of items.\n   */\n  private final int maxConsumptionLag;\n\n  ListBlobQueue(int maxSize, int consumeSetSize, int maxConsumptionLag)\n      throws InvalidConfigurationValueException {\n    this.maxSize = maxSize;\n    this.maxConsumptionLag = maxConsumptionLag;\n    this.consumeSetSize = consumeSetSize;\n\n    if (maxConsumptionLag >= maxSize) {\n      throw new InvalidConfigurationValueException(FS_AZURE_CONSUMER_MAX_LAG,\n          \"maxConsumptionLag should be lesser than maxSize\");\n    }\n  }\n\n  /** Mark the queue as failed.*/\n  void markProducerFailure(AzureBlobFileSystemException failure) {\n    failureFromProducer = failure;\n  }\n\n  /** Mark the queue as completed.*/\n  void complete() {\n    isCompleted = true;\n  }\n\n  /** Mark the consumption as failed.*/\n  synchronized void markConsumptionFailed() {\n    isConsumptionFailed = true;\n    notify();","sourceCodeStart":68,"sourceCodeEnd":104,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ListBlobQueue.java#L68-L104","documentation":"Error \"maxConsumptionLag should be lesser than maxSize\" thrown in apache/hadoop.","triggerScenarios":"ListBlobQueue is configured with maxConsumptionLag >= maxSize.","commonSituations":"See trigger scenarios.","solutions":["Configure maxConsumptionLag smaller than the listing queue maxSize.","Adjust fs.azure.list.blob.queue.* configuration values consistently."],"exampleFix":null,"handlingStrategy":null,"validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}