elastic/elasticsearch · error · IOException

failed to skip [{}] bytes while reading [{}]

Error message

failed to skip [{}] bytes while reading [{}]

What it means

Thrown by MMDBUtil.getDatabaseType when InputStream.skip does not skip the exact number of bytes requested (the last BUFFER_SIZE bytes of the file). Per the InputStream contract, skip may return fewer bytes than requested; here any shortfall is treated as a read failure of the mmdb tail. IOException propagates to the caller that reads database type for lookup wiring.

Source

Thrown at modules/ip-location/src/main/java/org/elasticsearch/ingest/geoip/MMDBUtil.java:48

    private static final int BUFFER_SIZE = 2048;

    /**
     * Read the database type from the database. We do this manually instead of relying on the built-in mechanism to avoid reading the
     * entire database into memory merely to read the type. This is especially important to maintain on master nodes where pipelines are
     * validated. If we read the entire database into memory, we could potentially run into low-memory constraints on such nodes where
     * loading this data would otherwise be wasteful if they are not also ingest nodes.
     *
     * @return the database type
     * @throws IOException if an I/O exception occurs reading the database type
     */
    public static String getDatabaseType(final Path database) throws IOException {
        final long fileSize = Files.size(database);
        try (InputStream in = Files.newInputStream(database)) {
            // read the last BUFFER_SIZE bytes (or the fileSize, whichever is smaller)
            final long skip = fileSize > BUFFER_SIZE ? fileSize - BUFFER_SIZE : 0;
            final long skipped = in.skip(skip);
            if (skipped != skip) {
                throw new IOException("failed to skip [" + skip + "] bytes while reading [" + database + "]");
            }
            final byte[] tail = new byte[BUFFER_SIZE];
            int read = 0;
            int actualBytesRead;
            do {
                actualBytesRead = in.read(tail, read, BUFFER_SIZE - read);
                read += actualBytesRead;
            } while (actualBytesRead > 0);

            // find the database_type header
            int metadataOffset = -1;
            int markerOffset = 0;
            for (int i = 0; i < tail.length; i++) {
                byte b = tail[i];

                if (b == DATABASE_TYPE_MARKER[markerOffset]) {
                    markerOffset++;
                } else {

View on GitHub (pinned to db6a809a66)

Solutions

1. Verify the file is a valid mmdb and is non-empty (check size and magic).
2. Re-download the database in case of truncation.
3. Ensure the file is fully written/flushed before getDatabaseType reads it.
4. If reading from a custom FileSystem, use a stream whose skip is reliable or read-and-discard instead.

void assertMmdbReadable(Path p) throws IOException {
    if (Files.size(p) == 0) throw new IOException("empty mmdb: " + p);
    try (InputStream in = Files.newInputStream(p)) {
        long skipped = 0, target = Math.max(0, Files.size(p) - BUFFER_SIZE);
        while (skipped < target) {
            long s = in.skip(target - skipped);
            if (s <= 0) throw new IOException("cannot skip to mmdb tail: " + p);
            skipped += s;
        }
    }
}

try {
    String type = MMDBUtil.getDatabaseType(path);
} catch (IOException e) {
    if (e.getMessage().startsWith("failed to skip")) {
        // re-download or supply a fully-written valid mmdb
    } else throw e;
}

When it happens

Trigger: getDatabaseType(path): fileSize > BUFFER_SIZE -> skip = fileSize - BUFFER_SIZE; skipped = in.skip(skip); skipped != skip -> throw.

Common situations: Very small or empty mmdb file where skip semantics behave oddly; a non-mmdb file fed to the geoip loader; a corrupt or truncated download; a special filesystem whose InputStream.skip under-delivers.

Related errors

• database type marker not found
• type must be UTF-8 string
• database_type too long [size indicator == {}]
• Unable to create reaper JAR output directory {}
• Failed to write unicast_hosts for {}

AI-assisted analysis of elastic/elasticsearch@db6a809a66 (2026-08-12). Data as JSON: /api/errors/fc7c91f08a9e94da. Report an issue: GitHub.