could not resolve to a temp directory for import batch files

Error message

could not resolve to a temp directory for import batch files

What it means

fhirio's tryFallbackToDataflowTempDirOrPanic panics when neither temp_location nor staging_location pipeline options can be resolved. The FHIR import connector needs a GCS temp/staging directory to stage batch NDJSON files for the $import operation, and without one it cannot proceed.

Solutions

Set the Dataflow pipeline option --temp_location=gs://your-bucket/temp when launching the pipeline.
If temp_location is intentionally absent, set --staging_location=gs://your-bucket/staging as the fallback.
Verify the option actually reaches the worker (check beam.PipelineOptions.Get("temp_location") semantics and your submission flags).
Pre-check in main: fail fast with a clear message if both options are empty before building the pipeline.

Example fix

// before
beam.Run(ctx, proj, "dataflow", p) // no temp_location

// after
// submit with: --temp_location=gs://my-bucket/temp
beam.Run(ctx, proj, "dataflow", p)

Defensive patterns

Strategy: validation

Validate before calling

if beam.PipelineOptions.Get("temp_location") == "" && beam.PipelineOptions.Get("staging_location") == "" {
    return errors.New("fhirio.Import requires --temp_location or --staging_location pipeline option")
}

Try / catch

defer func() {
    if r := recover(); r != nil {
        if strings.Contains(fmt.Sprint(r), "temp directory") {
            log.Fatalf("set --temp_location (Dataflow) before using fhirio.Import: %v", r)
        }
        panic(r)
    }
}()

Prevention

Always pass --temp_location=gs://bucket/path when launching Dataflow jobs using fhirio.
Fail fast in main() when required pipeline options are missing.
Ensure the GCS bucket exists and the service account can write to it.

When it happens

Trigger: Running fhirio.Import on Dataflow without setting the temp_location pipeline option, and with staging_location also unset (or empty strings in the pipeline options).

Common situations: Submitting a Dataflow job where --temp-location was forgotten; running with a runner that doesn't populate these options automatically; a typo like --temp_location vs --tempLocation in custom submission tooling.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d23e2b8210a7b49c. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/io/fhirio/import.go:258

			DeadLetterLocation: deadLetterDir,
			ContentStructure:   contentStructure,
		},
		batchFiles,
	)
	return failedResources, failedImportsDeadLetter
}

func tryFallbackToDataflowTempDirOrPanic() string {
	beam.PipelineOptions.LoadOptionsFromFlags(nil)
	if f := beam.PipelineOptions.Get("temp_location"); f != "" {
		return f
	}

	// temp_location is optional, so fallback to staging_location.
	if f := beam.PipelineOptions.Get("staging_location"); f != "" {
		return f
	}
	panic("could not resolve to a temp directory for import batch files")
}

View on GitHub (pinned to 12126d8942)