Griffin错误：java.lang.NoSuchMethodError

huangapple
未分类评论42阅读模式
英文:

Griffin Error : java.lang.NoSuchMethodError

问题

需要帮助。我正在尝试执行文档中提供的作业。

我系统中已安装以下版本：

  • Hadoop：3.2.0
  • Scala：2.13.1
  • Spark：3.0.0
  • Java：1.8.0
  • Hive：3.1.2

我正在提交 Spark 作业以使用 Griffin 运行数据质量检查。我遇到以下异常，请您帮忙解决。

2020-04-04 08:37:35,456 INFO cluster.YarnClientSchedulerBackend: SchedulerBackend is ready for scheduling beginning after waiting maxRegisteredResourcesWaitingTime: 30000000000(ns)
Exception in thread "main" java.lang.NoSuchMethodError: scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/collection/mutable/ArrayOps;
        at org.apache.griffin.measure.datasource.connector.batch.HiveBatchDataConnector.<init>(HiveBatchDataConnector.scala:47)
        at org.apache.griffin.measure.datasource.connector.DataConnectorFactory$$anonfun$getDataConnector$1.apply(DataConnectorFactory.scala:63)
        at org.apache.griffin.measure.datasource.connector.DataConnectorFactory$$anonfun$getDataConnector$1.apply(DataConnectorFactory.scala:62)
        at scala.util.Try$.apply(Try.scala:213)
        at org.apache.griffin.measure.datasource.connector.DataConnectorFactory$.getDataConnector(DataConnectorFactory.scala:61)
        at org.apache.griffin.measure.datasource.DataSourceFactory$$anonfun$1.apply(DataSourceFactory.scala:58)
        at org.apache.griffin.measure.datasource.DataSourceFactory$$anonfun$1.apply(DataSourceFactory.scala:57)
        // 更多异常信息...
英文:

Need help. I'm trying this job given in the documentation.

I have the following versions installed in my system:

  • Hadoop:3.2.0
  • Scala: 2.13.1
  • Spark: 3.0.0
  • Java: 1.8.0
  • Hive: 3.1.2

I'm submitting the spark job to run data quality using griffin. I'm getting below exception can you pls help me in this regard.

2020-04-04 08:37:35,456 INFO cluster.YarnClientSchedulerBackend: SchedulerBackend is ready for scheduling beginning after waiting maxRegisteredResourcesWaitingTime: 30000000000(ns)
Exception in thread &quot;main&quot; java.lang.NoSuchMethodError: scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/collection/mutable/ArrayOps;
        at org.apache.griffin.measure.datasource.connector.batch.HiveBatchDataConnector.&lt;init&gt;(HiveBatchDataConnector.scala:47)
        at org.apache.griffin.measure.datasource.connector.DataConnectorFactory$$anonfun$getDataConnector$1.apply(DataConnectorFactory.scala:63)
        at org.apache.griffin.measure.datasource.connector.DataConnectorFactory$$anonfun$getDataConnector$1.apply(DataConnectorFactory.scala:62)
        at scala.util.Try$.apply(Try.scala:213)
        at org.apache.griffin.measure.datasource.connector.DataConnectorFactory$.getDataConnector(DataConnectorFactory.scala:61)
        at org.apache.griffin.measure.datasource.DataSourceFactory$$anonfun$1.apply(DataSourceFactory.scala:58)
        at org.apache.griffin.measure.datasource.DataSourceFactory$$anonfun$1.apply(DataSourceFactory.scala:57)
        at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:245)
        at scala.collection.immutable.List.foreach(List.scala:392)
        at scala.collection.TraversableLike.flatMap(TraversableLike.scala:245)
        at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:242)
        at scala.collection.immutable.List.flatMap(List.scala:355)
        at org.apache.griffin.measure.datasource.DataSourceFactory$.org$apache$griffin$measure$datasource$DataSourceFactory$$getDataSource(DataSourceFactory.scala:57)
        at org.apache.griffin.measure.datasource.DataSourceFactory$$anonfun$getDataSources$1.apply(DataSourceFactory.scala:40)
        at org.apache.griffin.measure.datasource.DataSourceFactory$$anonfun$getDataSources$1.apply(DataSourceFactory.scala:38)
        at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:245)
        at scala.collection.immutable.List.foreach(List.scala:392)
        at scala.collection.TraversableLike.flatMap(TraversableLike.scala:245)
        at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:242)
        at scala.collection.immutable.List.flatMap(List.scala:355)
        at org.apache.griffin.measure.datasource.DataSourceFactory$.getDataSources(DataSourceFactory.scala:38)
        at org.apache.griffin.measure.launch.batch.BatchDQApp$$anonfun$run$1.apply$mcZ$sp(BatchDQApp.scala:75)
        at org.apache.griffin.measure.launch.batch.BatchDQApp$$anonfun$run$1.apply(BatchDQApp.scala:67)
        at org.apache.griffin.measure.launch.batch.BatchDQApp$$anonfun$run$1.apply(BatchDQApp.scala:67)
        at scala.util.Try$.apply(Try.scala:213)
        at org.apache.griffin.measure.launch.batch.BatchDQApp.run(BatchDQApp.scala:67)
        at org.apache.griffin.measure.Application$.main(Application.scala:88)
        at org.apache.griffin.measure.Application.main(Application.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
        at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:928)
        at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
        at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1007)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1016)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

答案1

得分: 0

java.lang.NoSuchMethodError: scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/collection/mutable/ArrayOps;

这绝对意味着您的应用程序中的某处存在Scala版本冲突。

在这里有一个非常类似的问题1。尝试按照那里提供的答案进行操作（将Scala版本降级为2.11.x）。


<details>
<summary>英文:</summary>

java.lang.NoSuchMethodError: scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/collection/mutable/ArrayOps;


It definitely means that you have conflicts in the Scala version somewhere in your application.

There is a very similar question [here][1]. Try to follow the answer provided there (downgrade Scala version to `2.11.x`).


  [1]: https://stackoverflow.com/questions/40328948/java-lang-nosuchmethoderror-scala-predef-refarrayops

</details>



# 答案2
**得分**: 0

我不认为任何Spark版本已经针对Scala 2.13进行了编译/发布。

<details>
<summary>英文:</summary>

I don&#39;t think any of the Spark versions have been compiled/released for Scala 2.13

</details>

huangapple
  • 本文由 发表于 2020年4月5日 11:39:36
  • 转载请务必保留本文链接：https://java.coder-hub.com/61037744.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定