{"record":{"id":"14cea8a44a43613b","repo":"apache/beam","slug":"use-of-withmaxbufferingduration-requires","errorCode":null,"errorMessage":"Use of withMaxBufferingDuration requires withUseStatefulBatches(true). Setting that automatically.","messagePattern":"Use of withMaxBufferingDuration requires withUseStatefulBatches\\(true\\)\\. Setting that automatically\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"sdks/java/io/elasticsearch/src/main/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.java","lineNumber":2522,"sourceCode":"        allowableResponseErrorTypes = new HashSet<>();\n      }\n\n      return builder().setAllowedResponseErrors(allowableResponseErrorTypes).build();\n    }\n\n    /**\n     * If using {@link BulkIO#withUseStatefulBatches}, this can be used to set a maximum elapsed\n     * time before buffered elements are emitted to Elasticsearch as a Bulk API request. If this\n     * config is not set, Bulk requests will not be issued until {@link BulkIO#getMaxBatchSize}\n     * number of documents have been buffered. This may result in higher latency in particular if\n     * your max batch size is set to a large value and your pipeline input is low volume.\n     *\n     * @param maxBufferingDuration the maximum duration to wait before sending any buffered\n     *     documents to Elasticsearch, regardless of maxBatchSize.\n     * @return the {@link BulkIO} with maximum buffering duration set\n     */\n    public BulkIO withMaxBufferingDuration(Duration maxBufferingDuration) {\n      LOG.warn(\n          \"Use of withMaxBufferingDuration requires withUseStatefulBatches(true). \"\n              + \"Setting that automatically.\");\n      return builder()\n          .setUseStatefulBatches(true)\n          .setMaxBufferingDuration(maxBufferingDuration)\n          .build();\n    }\n\n    /**\n     * Whether or not to use Stateful Processing to ensure bulk requests have the desired number of\n     * entities i.e. as close to the maxBatchSize as possible. By default without this feature\n     * enabled, Bulk requests will not contain more than maxBatchSize entities, but the lower bound\n     * of batch size is determined by Beam Runner bundle sizes, which may be as few as 1.\n     *\n     * @param useStatefulBatches true enables the use of Stateful Processing to ensure that batches\n     *     are as close to the maxBatchSize as possible.\n     * @return the {@link BulkIO} with Stateful Processing enabled or disabled\n     */","sourceCodeStart":2504,"sourceCodeEnd":2540,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/elasticsearch/src/main/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.java#L2504-L2540","documentation":"ElasticsearchIO's bulk batching with a max buffering duration is implemented on top of Beam's stateful processing, which requires stateful batches to be enabled. Calling withMaxBufferingDuration alone would produce a silently ignored setting, so the sink logs this warning and automatically enables withUseStatefulBatches(true) while applying the requested duration.","triggerScenarios":"Calling BulkIO.withMaxBufferingDuration(Duration) on an ElasticsearchIO write without having previously called withUseStatefulBatches(true).","commonSituations":"Developers tuning Elasticsearch bulk flush latency set a buffering timeout to avoid waiting for maxBatchSize to fill, forgetting that the timeout only works in stateful-batch mode.","solutions":["Explicitly call .withUseStatefulBatches(true) before .withMaxBufferingDuration(...) to make the dependency clear","Accept the automatic enabling — no action strictly required, but be aware stateful batches change pipeline execution","Review whether a max buffering duration is needed at all; the default batch-size-driven flushing may suffice"],"exampleFix":"// before\nElasticsearchIO.write().withConnectionConfiguration(cfg).withMaxBufferingDuration(Duration.standardSeconds(5))\n// after\nElasticsearchIO.write().withConnectionConfiguration(cfg).withUseStatefulBatches(true).withMaxBufferingDuration(Duration.standardSeconds(5))","handlingStrategy":"validation","validationCode":"if (maxBufferingDuration != null && !write.withUseStatefulBatches) { write = write.withUseStatefulBatches(true); } // ensure stateful batches precede withMaxBufferingDuration(...)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always chain withUseStatefulBatches(true) directly before withMaxBufferingDuration(...)","Centralize the ElasticsearchIO sink builder in a helper so buffering options are set consistently","Watch startup logs for automatic-setting warnings after pipeline upgrades"],"tags":["java","apache-beam","elasticsearch","configuration"],"backgroundTag":"conflicting-config-options","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}