apache/beam · warning · NoEstimationException

Cannot estimate the row count. All the sampled lines are emp

Error message

Cannot estimate the row count. All the sampled lines are empty

What it means

TextRowCountEstimator samples lines from files to estimate the row count of a text-based source. If no lines could be read (numberOfReadLines == 0) but sampling did not cover everything (sampledEverything == false), the estimator has no average line size to extrapolate from and cannot produce an estimate, so it throws NoEstimationException. This is a signal that the caller should fall back to another estimation strategy, not a data corruption error.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/io/TextRowCountEstimator.java:146

        int numberOfNonEmptyLines = 0;
        for (boolean more = reader.start(); more; more = reader.advance()) {
          numberOfNonEmptyLines += reader.getCurrent().trim().equals("") ? 0 : 1;
        }
        numberOfReadLines += numberOfNonEmptyLines;
        linesSize += (numberOfNonEmptyLines == 0) ? 0 : readingWindowSize;
      }
      long fileSize = metadata.sizeBytes();
      numberOfReadFiles += fileSize == 0 ? 0 : 1;
      totalFileSizes += fileSize;
    }

    if (numberOfReadLines == 0 && sampledEverything) {
      return 0d;
    }

    if (numberOfReadLines == 0) {
      throw new NoEstimationException(
          "Cannot estimate the row count. All the sampled lines are empty");
    }

    // This is total file sizes divided by average line size.
    return (double) totalFileSizes * numberOfReadLines / linesSize;
  }

  /** Builder for {@link org.apache.beam.sdk.io.TextRowCountEstimator}. */
  @AutoValue.Builder
  public abstract static class Builder {

    public abstract Builder setNumSampledBytesPerFile(long numSampledBytes);

    public abstract Builder setDirectoryTreatment(
        FileIO.ReadMatches.DirectoryTreatment directoryTreatment);

    public abstract Builder setCompression(Compression compression);

View on GitHub (pinned to 12126d8942)

Solutions

Check the input folder/files actually contain non-empty lines before estimating; if the data is genuinely empty, treat row count as 0 explicitly.
Catch NoEstimationException and fall back to a different estimation strategy (e.g. read the full count via a Count.glob job or use file sizes with a default average line length).
Increase the sampling size/number of lines in TextRowCountEstimator.Builder (e.g. withSampleNewLineProbability / withFileParsingTimeout) so at least one non-empty line is read.
Verify file encoding/newline conventions (e.g. files with only \r\n or only whitespace) are not causing every sampled line to be empty.

Example fix

// before
double rows = estimator.estimateRowCount(); // throws NoEstimationException on empty sample

// after
try {
  double rows = estimator.estimateRowCount();
} catch (NoEstimationException e) {
  double rows = 0d; // or fall back to another estimation method
}

Defensive patterns

Strategy: try-catch

Validate before calling

if (newDirectorySize(folder) == 0) { return 0d; } // short-circuit empty input before estimating

Try / catch

try {
  double rows = estimator.estimateRowCount();
} catch (NoEstimationException e) {
  double rows = fallBackToFullScanOrZero();
}

Prevention

Check that the input location contains non-empty files before running estimation.
Configure the estimator's sampling parameters so at least one non-empty line is always read.
Treat NoEstimationException as a signal to fall back, not as a failure.

When it happens

Trigger: Calling estimateRowCount on a TextRowCountEstimator configured over files/folder where every sampled line read was empty (zero-length) and the estimator did not sample everything (non-empty file content exists but sampling yielded no usable lines, e.g. all sampled reads returned empty strings or file sizes are zero while files are non-empty).

Common situations: Estimating row counts on folders containing only empty files or files full of blank lines; sampling configured with a maximen sample fraction/bytes that reads zero non-empty lines; using TextIO with needCreativeFileNameValidation or custom configuration where file size is 0 bytes; running on empty GCS/S3 folders that still exist.

Understand the failure class

Background: EmptyResultError / "no results found": when an API or scraper succeeds but returns zero rows — this error's family across 9 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d1e7475d9042f6de. Report an issue: GitHub.