{"record":{"id":"ee6fa0867cc81236","repo":"mlflow/mlflow","slug":"exception-header-with-string-representation-ra","errorCode":null,"errorMessage":"{exception_header} with string representation '{raw_artifact}' that is neither a valid path to a file nor a JSON string.","messagePattern":"(.+?) with string representation '(.+?)' that is neither a valid path to a file nor a JSON string\\.","errorType":"exception","errorClass":"MlflowException","httpStatus":null,"severity":"error","filePath":"mlflow/models/evaluation/artifacts.py","lineNumber":165,"sourceCode":"    exception_header = (\n        f\"Custom metric function '{custom_metric_tuple.name}' at index \"\n        f\"{custom_metric_tuple.index} in the `custom_metrics` parameter produced an \"\n        f\"artifact '{artifact_name}'\"\n    )\n\n    # Given a string, first see if it is a path. Otherwise, check if it is a JsonEvaluationArtifact\n    if isinstance(raw_artifact, str):\n        potential_path = pathlib.Path(raw_artifact)\n        if potential_path.exists():\n            raw_artifact = potential_path\n        else:\n            try:\n                json.loads(raw_artifact)\n                return _InferredArtifactProperties(\n                    from_path=False, type=JsonEvaluationArtifact, ext=\".json\"\n                )\n            except JSONDecodeError:\n                raise MlflowException(\n                    f\"{exception_header} with string representation '{raw_artifact}' that is \"\n                    f\"neither a valid path to a file nor a JSON string.\"\n                )\n\n    # Type inference based on the file extension\n    if isinstance(raw_artifact, pathlib.Path):\n        if not raw_artifact.exists():\n            raise MlflowException(f\"{exception_header} with path '{raw_artifact}' does not exist.\")\n        if not raw_artifact.is_file():\n            raise MlflowException(f\"{exception_header} with path '{raw_artifact}' is not a file.\")\n        if raw_artifact.suffix not in _EXT_TO_ARTIFACT_MAP:\n            raise MlflowException(\n                f\"{exception_header} with path '{raw_artifact}' does not match any of the supported\"\n                f\" file extensions: {', '.join(_EXT_TO_ARTIFACT_MAP.keys())}.\"\n            )\n        return _InferredArtifactProperties(\n            from_path=True, type=_EXT_TO_ARTIFACT_MAP[raw_artifact.suffix], ext=raw_artifact.suffix\n        )","sourceCodeStart":147,"sourceCodeEnd":183,"githubUrl":"https://github.com/mlflow/mlflow/blob/6a27f2decc0b76eb1b54af31849784addb357dbc/mlflow/models/evaluation/artifacts.py#L147-L183","documentation":"_infer_artifact_type_and_ext determines how a custom metric value passed to mlflow.evaluate should be logged as an artifact. If a string is neither an existing file path nor parseable JSON, MLflow raises this MlflowException because it cannot infer a serialization type. The {exception_header} prefix names the offending argument (e.g. 'Value ... for custom metric \"x\"').","triggerScenarios":"Returning a plain non-JSON string from a custom metric/float-castable value passed as an artifact, e.g. `pred = \"result file abc\"` or a string that is a path-like name but doesn't exist, and not a pathlib.Path — the JSON decode fallback then fails and this is raised.","commonSituations":"Returning free-form text (e.g. an LLM completion or error message) from a custom metric without specifying artifact_type; typos in file paths (string path pointing to a missing file); forgetting to wrap content in json.dumps.","solutions":["Serialize the string as JSON before returning it (json.dumps(value)).","Pass a real pathlib.Path to an existing supported file instead of a string path.","Return a typed EvaluationArtifact instance (e.g. JsonEvaluationArtifact) to skip inference.","If the value is genuinely text, wrap it as {\"text\": value} and log via a JSON artifact."],"exampleFix":"// before\n// def my_metric(...):\n//     return \"model said: hello\"\n// after\n// import json\n// def my_metric(...):\n//     return json.dumps({\"text\": \"model said: hello\"})","handlingStrategy":"validation","validationCode":"import json, pathlib\ndef inferable(v):\n    if isinstance(v, pathlib.Path):\n        return v.exists()\n    if isinstance(v, str):\n        return pathlib.Path(v).exists()\n    try:\n        json.loads(v)\n        return True\n    except Exception:\n        return False\nassert inferable(value), f\"custom metric return {value!r} is neither an existing path nor JSON\"","typeGuard":"import json, pathlib\ndef is_json_str(v: str) -> bool:\n    try:\n        json.loads(v)\n        return True\n    except json.JSONDecodeError:\n        return False","tryCatchPattern":"from mlflow.exceptions import MlflowException\ntry:\n    result = mlflow.evaluate(...)\nexcept MlflowException as e:\n    if \"neither a valid path to a file nor a JSON string\" in str(e):\n        raise SystemExit(\"Fix custom metric return: wrap strings with json.dumps or return a real Path\")\n    raise","preventionTips":["Always json.dumps plain strings returned from custom metrics","Return pathlib.Path objects (not str) when referencing files","Prefer returning typed EvaluationArtifact instances to skip inference entirely"],"tags":["evaluation","artifacts","type-inference","validation"],"backgroundTag":"artifact-type-inference-failed","analyzedSha":"6a27f2decc0b76eb1b54af31849784addb357dbc","analyzedAt":"2026-08-29T20:54:51.419Z","schemaVersion":2},"datasetVersion":"2026-08-29T22:17:34.462Z"}