apache/beam · error

expected float64, got data of type %T instead

Error message

expected float64, got data of type %T instead

What it means

Dataflow counter metric values arrive as generic decoded JSON, and extractCounterValue asserts they are float64 before converting to int64. This error means the counter's scalar value had a different Go type (string, nil, nested object, etc.), so it cannot be interpreted as a numeric counter.

Solutions

1. Handle JSON null/missing values before calling extractCounterValue and default to 0 or skip
2. Check whether the metric is a distribution and route it to extractDistributionValue instead
3. Guard with a type assertion and log the raw %T payload for debugging

Example fix

// before
val, err := extractCounterValue(raw)
// after
if raw == nil {
    raw = float64(0)
}
val, err := extractCounterValue(raw)

if v, ok := raw.(float64); !ok {
    log.Printf("counter %v: non-numeric value %T, skipping", name, raw)
    return
}

func isCounterValue(obj any) (int64, bool) {
    v, ok := obj.(float64)
    return int64(v), ok
}

val, err := extractCounterValue(raw)
if err != nil {
    log.Printf("bad counter value: %v", err)
    continue
}

When it happens

Trigger: groupByType calls extractCounterValue on a MetricUpdate value that is not a JSON number — e.g. a counter whose value is null (no updates), a string-encoded number, or a distributed-metric payload routed to the counter path.

Common situations: Counters reported with no value yet (null) in early job stages; Dataflow API returning values with unexpected shapes; misrouted distribution updates handled by the counter extractor.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• could not find the internal step name
• could not translate the internal step name
• failed to get metrics
• bad I
• bad KV

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/304e36fc5c05bad8. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/runners/dataflow/dataflowlib/metrics.go:101

			break
		}
	}
	if userStepName == "" {
		return metrics.StepKey{}, fmt.Errorf("could not translate the internal step name %v", stepName)
	}

	namespace := metric.Name.Context["namespace"]
	if namespace == "" {
		namespace = "dataflow/v1b3"
	}

	return metrics.StepKey{Step: userStepName, Name: metric.Name.Name, Namespace: namespace}, nil
}

func extractCounterValue(obj any) (int64, error) {
	v, ok := obj.(float64)
	if !ok {
		return -1, fmt.Errorf("expected float64, got data of type %T instead", obj)
	}
	return int64(v), nil
}

func extractDistributionValue(obj any) (metrics.DistributionValue, error) {
	m := obj.(map[string]any)
	propertiesToVisit := []string{"count", "sum", "min", "max"}
	var values [4]int64

	for i, p := range propertiesToVisit {
		v, ok := m[p].(float64)
		if !ok {
			return metrics.DistributionValue{}, fmt.Errorf("expected float64, got data of type %T instead", m[p])
		}
		values[i] = int64(v)
	}
	return metrics.DistributionValue{Count: values[0], Sum: values[1], Min: values[2], Max: values[3]}, nil
}

View on GitHub (pinned to 12126d8942)