What causes "error extracting from asts: %v" in apache/beam?

Read the wrapped cause: fix the underlying compile/type error in the input Go files first.

error extracting from asts

Error message

error extracting from asts: %v

What it means

starcgen.Generate wraps any error returned by e.FromAsts (the extraction step that converts parsed Go ASTs into the generator's intermediate representation) as 'error extracting from asts: %v'. starcgen generates Beam schema encoders/decoders from Go source files, and this error means the source could not be analyzed for type information.

Solutions

Read the wrapped cause: fix the underlying compile/type error in the input Go files first.
Run `go build ./...` on the package being generated to confirm it compiles standalone.
Ensure all imported packages are present in the module (go mod tidy).
Check the debug output file written by Generate for the exact failing type/AST node.
If a specific type construct is unsupported, simplify it or upgrade the Beam SDK.

Example fix

// before (input references undefined type)
type Row struct { X UnknownType }
// after
type Row struct { X string } // ensure the referenced package is imported and exists

Defensive patterns

Strategy: validation

Validate before calling

if err := exec.Command("go", "build", "./...").Run(); err != nil { return fmt.Errorf("package must compile before starcgen: %w", err) }

Try / catch

err := gen.Generate(w, ...)
if err != nil && strings.Contains(err.Error(), "error extracting from asts") {
	log.Fatalf("fix source type errors: %v", err)
}

Prevention

Run `go build ./...` (or `go vet`) on input packages before generation.
Run `go mod tidy` so all imported packages resolve.
Keep generated-source inputs simple; avoid constructs unsupported by the extractor.

When it happens

Trigger: Running starcgen with --inputs pointing at Go files that fail type-checking: undefined types/packages, import errors, syntax-adjacent issues, or types outside the 'source' importer's reach.

Common situations: Generating from files that don't compile (missing dependencies in the module), referencing external packages not resolvable by importer.For("source"), or running starcgen on files with generics/types unsupported by the extractor.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4613b5a5388fe080. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/cmd/starcgen/starcgen.go:83

	ids         = flag.String("identifiers", "", "comma separated list of package local identifiers for which to generate code")
	debug       = flag.Bool("debug", false, "print out a debugging header in the shim file to help diagnose errors")
)

// Generate takes the typechecked inputs, and generates the shim file for the relevant
// identifiers.
func Generate(w io.Writer, filename, pkg string, ids []string, fset *token.FileSet, files []*ast.File) error {
	e := starcgenx.NewExtractor(pkg)
	e.Ids = ids
	e.Debug = *debug

	// Importing from source should work in most cases.
	imp := importer.For("source", nil)
	if err := e.FromAsts(imp, fset, files); err != nil {
		// Always print out the debugging info to the file.
		if _, errw := w.Write(e.Bytes()); errw != nil {
			return fmt.Errorf("error writing debug data to file after err %v:%v", err, errw)
		}
		err = fmt.Errorf("error extracting from asts: %v", err)
		e.Printf("%v", err)
		return err
	}
	data := e.Generate(filename)
	if err := write(w, []byte(license)); err != nil {
		return err
	}
	return write(w, data)
}

func write(w io.Writer, data []byte) error {
	n, err := w.Write(data)
	if err != nil && n < len(data) {
		return fmt.Errorf("short write of data got %d, want %d", n, len(data))
	}
	return err
}

View on GitHub (pinned to 12126d8942)