我是scala的新手,我正在尝试使用Scala和Java创建一个混合项目.但是,当我运行测试代码时,我遇到了一些问题.当我运行测试时,我收到一个错误
和我的pom.xml如下:
2.12.2 org.scala-lang scala-library ${scala.version} org.scala-lang scala-compiler ${scala.version} org.scala-lang scala-reflect ${scala.version} org.apache.spark spark-streaming_2.11 2.1.1 org.apache.spark spark-streaming-kafka-0-10_2.11 2.1.1 org.scala-tools maven-scala-plugin 2.15.2 compile compile compile test-compile testCompile test-compile process-resources compile maven-compiler-plugin 1.5
我的代码如下:
class BptConsumer { def consumeLogevent(): Unit ={ val conf = new SparkConf().setMaster("local[2]").setAppName("PVStatistics"); val ssc = new StreamingContext(conf,Seconds(5)); val kafkaParams = Map[String, Object]( "bootstrap.servers" -> "172.20.13.196:9092", "key.deserializer" -> classOf[StringDeserializer], "value.deserializer" -> classOf[StringDeserializer], "group.id" -> "1", "auto.offset.reset" -> "latest", "enable.auto.commit" -> (false: java.lang.Boolean) ) val topics = Array("fd-blogs-tst") val stream = KafkaUtils.createDirectStream[String, String]( ssc, PreferConsistent, Subscribe[String, String](topics, kafkaParams) ) /*val rdd = stream.transform(x=>RDD[String]);*/ val lines = stream.map(record => (record.key,record.value)) lines.print(); ssc.start(); ssc.awaitTermination(); } }
有人可以帮助我找到问题吗?
您正在使用Scala 2.12.2和使用Scala 2.11构建的Spark库.将Scala版本更改为2.11版本:
2.11.11