org.apache.spark.sql.types.Decimal cannot be cast to org.apache.spark.unsafe.types.UTF8String ___ using Spark / Java

huangapple 未分类评论42阅读模式
英文:

org.apache.spark.sql.types.Decimal cannot be cast to org.apache.spark.unsafe.types.UTF8String ___ using Spark / Java

问题

我有以下数据帧:

+-------------+-----------------+------------------+
|longitude    |latitude         |geom              |
+-------------+-----------------+------------------+
|-7.07378166  |33.826661        [00 00 00 00 01 0..|
|-7.5952683   |33.544191        [00 00 00 00 01 0..|                  
+-------------+-----------------+------------------+

我正在使用以下代码:

Dataset<Row> result_f = sparkSession.sql("select * from data_f where ST_WITHIN(ST_PointFromText(CAST(data_f.latitude_f AS Decimal(24,20)), CAST(data_f.longitude_f AS Decimal(24,20))),geom)");
result_f.show();

当我执行它时,我得到以下异常:

java.lang.ClassCastException: org.apache.spark.sql.types.Decimal cannot be cast to org.apache.spark.unsafe.types.UTF8String
	at org.apache.spark.sql.geosparksql.expressions.ST_PointFromText.eval(Constructors.scala:44)
	at org.apache.spark.sql.geosparksql.expressions.ST_Within.eval(Predicates.scala:104)
	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificPredicate.And_0$(Unknown Source)
	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificPredicate.eval(Unknown Source)
	at org.apache.spark.sql.execution.joins.CartesianProductExec$$anonfun$doExecute$1$$anonfun$2.apply(CartesianProductExec.scala:89)
	at org.apache.spark.sql.execution.joins.CartesianProductExec$$anonfun$doExecute$1$$anonfun$2.apply(CartesianProductExec.scala:88)

你有任何想法吗?

我需要你的帮助。

谢谢

英文:

I have the following dataframe :

+-------------+-----------------+------------------+
|longitude    |latitude         |geom              |
+-------------+-----------------+------------------+
|-7.07378166  |33.826661        [00 00 00 00 01 0..|
|-7.5952683   |33.544191        [00 00 00 00 01 0..|                  
+-------------+-----------------+------------------+

I'm using the following code :

 Dataset&lt;Row&gt; result_f = sparkSession.sql(&quot;select * from data_f where ST_WITHIN(ST_PointFromText(CAST(data_f.latitude_f AS Decimal(24,20)), CAST(data_f.longitude_f AS Decimal(24,20))),geom)&quot;);
    result_f.show();

I get the following exception when I execute it :

java.lang.ClassCastException: org.apache.spark.sql.types.Decimal cannot be cast to org.apache.spark.unsafe.types.UTF8String
at org.apache.spark.sql.geosparksql.expressions.ST_PointFromText.eval(Constructors.scala:44)
at org.apache.spark.sql.geosparksql.expressions.ST_Within.eval(Predicates.scala:104)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificPredicate.And_0$(Unknown Source)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificPredicate.eval(Unknown Source)
at org.apache.spark.sql.execution.joins.CartesianProductExec$$anonfun$doExecute$1$$anonfun$2.apply(CartesianProductExec.scala:89)
at org.apache.spark.sql.execution.joins.CartesianProductExec$$anonfun$doExecute$1$$anonfun$2.apply(CartesianProductExec.scala:88)

Have you any idea ?

I need your help .

Thank you

huangapple
  • 本文由 发表于 2020年7月24日 21:04:59
  • 转载请务必保留本文链接:https://java.coder-hub.com/63074158.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定