{"record":{"id":"d2d069d9a6a54c38","repo":"xai-org/x-algorithm","slug":"job-config-specified-entitytype-semanticcore-but","errorCode":null,"errorMessage":"Job config specified EntityType.SemanticCore, but non-semantic core entity was found.","messagePattern":"Job config specified EntityType\\.SemanticCore, but non-semantic core entity was found\\.","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"simclusters/simclusters_v2/scalding/embedding/LocaleEntitySimClustersEmbeddingsJob.scala","lineNumber":330,"sourceCode":"    semanticCoreEntityIdsToKeep: Option[TypedPipe[Long]],\n    applyLogTransform: Boolean = false\n  ): TypedPipe[(UserId, (Entity, Double))] =\n    jobConfig.entityType match {\n      case EntityType.SemanticCore =>\n        semanticCoreEntityIdsToKeep match {\n          case Some(entityIdsToKeep) =>\n            getEntityUserMatrix(entityRealGraphSource, jobConfig.halfLife, EntityType.SemanticCore)\n              .map {\n                case (entity, (userId, score)) =>\n                  entity match {\n                    case Entity.SemanticCore(SemanticCoreEntity(entityId, _)) =>\n                      if (applyLogTransform) {\n                        (entityId, (userId, (entity, Math.log(score + 1))))\n                      } else {\n                        (entityId, (userId, (entity, score)))\n                      }\n                    case _ =>\n                      throw new IllegalArgumentException(\n                        \"Job config specified EntityType.SemanticCore, but non-semantic core entity was found.\")\n                  }\n              }.hashJoin(entityIdsToKeep.asKeys).values.map {\n                case ((userId, (entity, score)), _) => (userId, (entity, score))\n              }\n          case _ =>\n            getEntityUserMatrix(entityRealGraphSource, jobConfig.halfLife, EntityType.SemanticCore)\n              .map { case (entity, (userId, score)) => (userId, (entity, score)) }\n        }\n      case EntityType.Hashtag =>\n        getEntityUserMatrix(entityRealGraphSource, jobConfig.halfLife, EntityType.Hashtag)\n          .map { case (entity, (userId, score)) => (userId, (entity, score)) }\n      case _ =>\n        throw new IllegalArgumentException(\n          s\"Argument [--entity-type] must be provided. Supported options [${EntityType.SemanticCore.name}, ${EntityType.Hashtag.name}]\")\n    }\n\n  def toSimClustersEmbeddingId(","sourceCodeStart":312,"sourceCodeEnd":348,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/simclusters/simclusters_v2/scalding/embedding/LocaleEntitySimClustersEmbeddingsJob.scala#L312-L348","documentation":"While building the user-entity matrix, the job expects every entity to be an Entity.SemanticCore when entityType == SemanticCore. Encountering any other Entity variant (Hashtag, TweetId, UserId) hits the catch-all and throws, protecting downstream InternalId.SemanticCore conversion from ClassCastException-like failures.","triggerScenarios":"Running with --entity-type SemanticCore while the input entity stream contains Entity.Hashtag or other non-SemanticCore entities (dirty upstream data or wrong source selection).","commonSituations":"Mixing entity sources in the RealGraph/input data; upstream schema change emitting hashtags into the semantic-core stream; running the hashtag pipeline with the semantic-core flag.","solutions":["Filter the input to only SemanticCore entities before this stage","Verify --entity-type matches the actual entity source being read","Add a reprocessing/validation job upstream to separate entity types"],"exampleFix":"// before\nuserEntityMatrix.flatMap {\n  case (userId, (entity: Entity.SemanticCore, score)) => Some(...)\n  case _ => throw new IllegalArgumentException(\"Job config specified EntityType.SemanticCore, but non-semantic core entity was found.\")\n}\n\n// after\nuserEntityMatrix.flatMap {\n  case (userId, (entity: Entity.SemanticCore, score)) => Some(...)\n  case _ =>\n    println(s\"Skipping non-semantic-core entity\")\n    None // or log & count instead of failing the whole job\n}","handlingStrategy":"type-guard","validationCode":"val semanticCoreOnly = input.filter { case (_, (e: Entity, _)) => e.isInstanceOf[Entity.SemanticCore] }","typeGuard":"def isSemanticCore(e: Entity): Boolean = e match {\n  case _: Entity.SemanticCore => true\n  case _ => false\n}","tryCatchPattern":"catch { case e: IllegalArgumentException if e.getMessage.contains(\"SemanticCore\") => log.error(\"Non-semantic-core entity in input; check upstream source\", e); sys.exit(2) }","preventionTips":["Filter the input stream by entity type before joining","Run a data-quality job to assert entity-type homogeneity before this pipeline"],"tags":["scala","simclusters","data-validation","entity-type","scalding"],"backgroundTag":"data-shape-validation-failed","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}