What causes "lambda memory must be at least 1GB" in quickwit-oss/quickwit?

Raise the Lambda memory setting to at least 1024MB (AWS console: Configuration > General configuration > Memory; CLI: aws lambda update-function-configuration --memory-size 2048).

quickwit-oss/quickwit · error · anyhow::Error

lambda memory must be at least 1GB

Error message

lambda memory must be at least 1GB

What it means

try_searcher_config_from_env reads AWS_LAMBDA_FUNCTION_MEMORY_SIZE from the Lambda runtime environment and requires the function to be configured with at least 1GB of memory. The warmup memory budget is derived by subtracting 500MiB of overhead, so less than 1GB would leave too little room for the searcher to operate. The anyhow::ensure! aborts startup otherwise.

Source

Thrown at quickwit/quickwit-lambda-server/src/context.rs:56

        let storage_resolver = StorageResolver::configured(&Default::default());

        Ok(Self {
            searcher_context,
            storage_resolver,
        })
    }
}

/// Create a Lambda-optimized searcher config based on the `AWS_LAMBDA_FUNCTION_MEMORY_SIZE`
/// environment variable.
fn try_searcher_config_from_env() -> anyhow::Result<SearcherConfig> {
    let lambda_memory_mib: u64 = quickwit_common::get_from_env_opt(
        "AWS_LAMBDA_FUNCTION_MEMORY_SIZE",
        /* sensitive */ false,
    )
    .context("could not get aws lambda function memory size from ENV")?;
    let lambda_memory = ByteSize::mib(lambda_memory_mib);
    anyhow::ensure!(
        lambda_memory >= ByteSize::gib(1u64),
        "lambda memory must be at least 1GB"
    );
    let warmup_memory_budget = ByteSize::b(lambda_memory.as_u64() - ByteSize::mib(500).as_u64());

    let mut searcher_config = SearcherConfig::default();
    searcher_config.max_num_concurrent_split_searches = 20;
    searcher_config.warmup_memory_budget = warmup_memory_budget;
    searcher_config.fast_field_cache = CacheConfig::no_cache();
    searcher_config.split_footer_cache = CacheConfig::no_cache();
    searcher_config.predicate_cache = CacheConfig::no_cache();
    searcher_config.partial_request_cache = CacheConfig::no_cache();
    Ok(searcher_config)
}

View on GitHub (pinned to a39730c5cd)

Solutions

Raise the Lambda memory setting to at least 1024MB (AWS console: Configuration > General configuration > Memory; CLI: aws lambda update-function-configuration --memory-size 2048).
In your IaC template, set MemorySize >= 1024 for the function resource.
If testing locally, set AWS_LAMBDA_FUNCTION_MEMORY_SIZE=2048 in the environment before running the handler.

Example fix

// before (SAM template)
MemorySize: 512
// after
MemorySize: 4096

Defensive patterns

Strategy: validation

Validate before calling

let mem: u64 = std::env::var("AWS_LAMBDA_FUNCTION_MEMORY_SIZE")
    .ok()
    .and_then(|v| v.parse().ok())
    .unwrap_or(0);
assert!(mem >= 1024, "Lambda memory must be >= 1024MB, got {}", mem);

Prevention

Set MemorySize >= 1024 (preferably 2048-4096) in your Lambda IaC template.
Add a deployment-time check that reads the function configuration after create/update.
When running locally, export AWS_LAMBDA_FUNCTION_MEMORY_SIZE explicitly.

When it happens

Trigger: Booting the quickwit-lambda-server binary inside a Lambda configured with AWS_LAMBDA_FUNCTION_MEMORY_SIZE below 1024 MiB (e.g. 512MB or 128MB in the function configuration or IaC template).

Common situations: Deploying the searcher Lambda with the AWS console default (128MB) or a cost-optimized small memory setting; SAM/CDK/Terraform templates that never raised memory_size for the function.

Understand the failure class

Background: "is not a valid" / "Invalid ... value" environment variable errors: how libraries validate env vars and what to do when they reject yours — this error's family across 48 libraries.

Related errors

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/7ca39ca2aed1b54d. Report an issue: GitHub.