{"record":{"id":"d1e7475d9042f6de","repo":"apache/beam","slug":"cannot-estimate-the-row-count-all-the-sampled-lines-are","errorCode":null,"errorMessage":"Cannot estimate the row count. All the sampled lines are empty","messagePattern":"Cannot estimate the row count\\. All the sampled lines are empty","errorType":"exception","errorClass":"NoEstimationException","httpStatus":null,"severity":"warning","filePath":"sdks/java/core/src/main/java/org/apache/beam/sdk/io/TextRowCountEstimator.java","lineNumber":146,"sourceCode":"\n        int numberOfNonEmptyLines = 0;\n        for (boolean more = reader.start(); more; more = reader.advance()) {\n          numberOfNonEmptyLines += reader.getCurrent().trim().equals(\"\") ? 0 : 1;\n        }\n        numberOfReadLines += numberOfNonEmptyLines;\n        linesSize += (numberOfNonEmptyLines == 0) ? 0 : readingWindowSize;\n      }\n      long fileSize = metadata.sizeBytes();\n      numberOfReadFiles += fileSize == 0 ? 0 : 1;\n      totalFileSizes += fileSize;\n    }\n\n    if (numberOfReadLines == 0 && sampledEverything) {\n      return 0d;\n    }\n\n    if (numberOfReadLines == 0) {\n      throw new NoEstimationException(\n          \"Cannot estimate the row count. All the sampled lines are empty\");\n    }\n\n    // This is total file sizes divided by average line size.\n    return (double) totalFileSizes * numberOfReadLines / linesSize;\n  }\n\n  /** Builder for {@link org.apache.beam.sdk.io.TextRowCountEstimator}. */\n  @AutoValue.Builder\n  public abstract static class Builder {\n\n    public abstract Builder setNumSampledBytesPerFile(long numSampledBytes);\n\n    public abstract Builder setDirectoryTreatment(\n        FileIO.ReadMatches.DirectoryTreatment directoryTreatment);\n\n    public abstract Builder setCompression(Compression compression);\n","sourceCodeStart":128,"sourceCodeEnd":164,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/core/src/main/java/org/apache/beam/sdk/io/TextRowCountEstimator.java#L128-L164","documentation":"TextRowCountEstimator samples lines from files to estimate the row count of a text-based source. If no lines could be read (numberOfReadLines == 0) but sampling did not cover everything (sampledEverything == false), the estimator has no average line size to extrapolate from and cannot produce an estimate, so it throws NoEstimationException. This is a signal that the caller should fall back to another estimation strategy, not a data corruption error.","triggerScenarios":"Calling estimateRowCount on a TextRowCountEstimator configured over files/folder where every sampled line read was empty (zero-length) and the estimator did not sample everything (non-empty file content exists but sampling yielded no usable lines, e.g. all sampled reads returned empty strings or file sizes are zero while files are non-empty).","commonSituations":"Estimating row counts on folders containing only empty files or files full of blank lines; sampling configured with a maximen sample fraction/bytes that reads zero non-empty lines; using TextIO with needCreativeFileNameValidation or custom configuration where file size is 0 bytes; running on empty GCS/S3 folders that still exist.","solutions":["Check the input folder/files actually contain non-empty lines before estimating; if the data is genuinely empty, treat row count as 0 explicitly.","Catch NoEstimationException and fall back to a different estimation strategy (e.g. read the full count via a Count.glob job or use file sizes with a default average line length).","Increase the sampling size/number of lines in TextRowCountEstimator.Builder (e.g. withSampleNewLineProbability / withFileParsingTimeout) so at least one non-empty line is read.","Verify file encoding/newline conventions (e.g. files with only \\r\\n or only whitespace) are not causing every sampled line to be empty."],"exampleFix":"// before\ndouble rows = estimator.estimateRowCount(); // throws NoEstimationException on empty sample\n\n// after\ntry {\n  double rows = estimator.estimateRowCount();\n} catch (NoEstimationException e) {\n  double rows = 0d; // or fall back to another estimation method\n}","handlingStrategy":"try-catch","validationCode":"if (newDirectorySize(folder) == 0) { return 0d; } // short-circuit empty input before estimating","typeGuard":null,"tryCatchPattern":"try {\n  double rows = estimator.estimateRowCount();\n} catch (NoEstimationException e) {\n  double rows = fallBackToFullScanOrZero();\n}","preventionTips":["Check that the input location contains non-empty files before running estimation.","Configure the estimator's sampling parameters so at least one non-empty line is always read.","Treat NoEstimationException as a signal to fall back, not as a failure."],"tags":["java","apache-beam","io","row-count-estimation"],"backgroundTag":"empty-result-set","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}