Skip to content

Commit d5c672a

Browse files
dzlabhuaxingao
authored andcommitted
[SPARK-32315][ML] Provide an explanation error message when calling require
### What changes were proposed in this pull request? Small improvement in the error message shown to user https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/mllib/util/MLUtils.scala#L537-L538 ### Why are the changes needed? The current behavior is an exception is thrown but without any specific details on the cause ``` Caused by: java.lang.IllegalArgumentException: requirement failedCaused by: java.lang.IllegalArgumentException: requirement failed at scala.Predef$.require(Predef.scala:212) at org.apache.spark.mllib.util.MLUtils$.fastSquaredDistance(MLUtils.scala:508) at org.apache.spark.mllib.clustering.EuclideanDistanceMeasure$.fastSquaredDistance(DistanceMeasure.scala:232) at org.apache.spark.mllib.clustering.EuclideanDistanceMeasure.isCenterConverged(DistanceMeasure.scala:190) at org.apache.spark.mllib.clustering.KMeans$$anonfun$runAlgorithm$4.apply(KMeans.scala:336) at org.apache.spark.mllib.clustering.KMeans$$anonfun$runAlgorithm$4.apply(KMeans.scala:334) at scala.collection.MapLike$MappedValues$$anonfun$foreach$3.apply(MapLike.scala:245) at scala.collection.MapLike$MappedValues$$anonfun$foreach$3.apply(MapLike.scala:245) at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733) at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:130) at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:130) at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:236) at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40) at scala.collection.mutable.HashMap.foreach(HashMap.scala:130) at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:732) at scala.collection.MapLike$MappedValues.foreach(MapLike.scala:245) at org.apache.spark.mllib.clustering.KMeans.runAlgorithm(KMeans.scala:334) at org.apache.spark.mllib.clustering.KMeans.run(KMeans.scala:251) at org.apache.spark.mllib.clustering.KMeans.run(KMeans.scala:233) ``` ### Does this PR introduce _any_ user-facing change? Yes, this PR adds an explanation message to be shown to user when requirement check is not meant ### How was this patch tested? manually Closes apache#29115 from dzlab/patch/SPARK-32315. Authored-by: dzlab <[email protected]> Signed-off-by: Huaxin Gao <[email protected]>
1 parent c1f160e commit d5c672a

File tree

1 file changed

+4
-2
lines changed

1 file changed

+4
-2
lines changed

mllib/src/main/scala/org/apache/spark/mllib/util/MLUtils.scala

+4-2
Original file line numberDiff line numberDiff line change
@@ -534,8 +534,10 @@ object MLUtils extends Logging {
534534
norm2: Double,
535535
precision: Double = 1e-6): Double = {
536536
val n = v1.size
537-
require(v2.size == n)
538-
require(norm1 >= 0.0 && norm2 >= 0.0)
537+
require(v2.size == n,
538+
s"Both vectors should have same length, found v1 is $n while v2 is ${v2.size}")
539+
require(norm1 >= 0.0 && norm2 >= 0.0,
540+
s"Both norms should be greater or equal to 0.0, found norm1=$norm1, norm2=$norm2")
539541
var sqDist = 0.0
540542
/*
541543
* The relative error is

0 commit comments

Comments
 (0)