Skip to content

Mllib test failed #112

@jiazhentim

Description

@jiazhentim

There many failed workloads in Mllib test.
Such as glm-regression. The following is the error log. I use spark 1.3.0 Can anyone help to find the
reason?

16/07/23 10:50:43 INFO SparkDeploySchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.0
Exception in thread "main" java.lang.NoClassDefFoundError: org.apache.spark.ml.attribute.Attribute
at mllib.perf.GLMRegressionTest.createInputData(MLAlgorithmTests.scala:119)
at mllib.perf.TestRunner$$anonfun$2.apply(TestRunner.scala:67)
at mllib.perf.TestRunner$$anonfun$2.apply(TestRunner.scala:66)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.immutable.Range.foreach(Range.scala:141)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at mllib.perf.TestRunner$.main(TestRunner.scala:66)
at mllib.perf.TestRunner.main(TestRunner.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
at java.lang.reflect.Method.invoke(Method.java:620)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:569)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:166)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:189)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:110)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.ml.attribute.Attribute
at java.net.URLClassLoader.findClass(URLClassLoader.java:600)
at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:786)
at java.lang.ClassLoader.loadClass(ClassLoader.java:764)
at java.lang.ClassLoader.loadClass(ClassLoader.java:741)
... 19 more

Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties

Java options: -Dspark.storage.memoryFraction=0.66 -Dspark.serializer=org.apache.spark.serializer.JavaSerializer -Dspark.locality.wait=60000000 -Dspark.shuffle.manager=SORT
Options: glm-regression --num-trials=10 --inter-trial-wait=3 --num-partitions=64 --random-seed=5 --num-examples=500000 --num-iterations=20 --optimizer=auto --reg-type=elastic-net --elastic-net-param=0.0 --reg-param=0.01 --feature-noise=1.0 --step-size=0.0 --label-noise=0.1 --intercept=0.2 --loss=l2 --num-features=10000

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions