sparkstreaming 写数据到 phoenix报错问题

tf.foreachRDD( rdd => { 
rdd.saveAsTextFile("hdfs://hacluster/user/comms/test") 
val str1 = "NeCpuState" 
val str2 = "DeviceID" 
rdd.filter(!_.contains(str1)).filter(!_.contains(str2)).map(_.replaceAll("\"", "")).map(_.split(",")).filter(_.length == 7) .foreach(line => { 
val row = line(0).substring(3).reverse + MD5Util.stringMD5(UUID.randomUUID().toString) 
val DeviceID = line(1) 
val DeviceName = line(2) 
val CollectionTime = line(3) 
val GranularityPeriod = line(4) 
val cpuUsage = line(5) 
val sql = "upsert into TEST (ROW,DEVICEID,DEVICENAME,COLLECTIONTIME,GRANULARITYPERIOD,CPUUSAGE) values (" + row + "," + DeviceID + "," + DeviceName + "," + CollectionTime + "," + GranularityPeriod + "," + cpuUsage + ")" 
 
execute_sql(sql) }) }) 
ssc } 
 
def execute_sql(sql: String): Unit ={ 
val conn = DriverManager.getConnection("jdbc:phoenix:190.15.119.5,190.15.119.6,190.15.119.7") 
val preStat = conn.prepareStatement(sql) 
preStat.executeUpdate() 
conn.commit() 
preStat.close() 
conn.close() }
 
报错 
 
Caused by: java.io.IOException: java.lang.reflect.InvocationTargetException
        at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)
        at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:448)
        at org.apache.hadoop.hbase.client.ConnectionManager.createConnectionInternal(ConnectionManager.java:357)
        at org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:144)
        at org.apache.phoenix.query.HConnectionFactory$HConnectionFactoryImpl.createConnection(HConnectionFactory.java:47)
        at org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(ConnectionQueryServicesImpl.java:286)
        ... 25 more
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
        ... 30 more
Caused by: java.lang.UnsupportedOperationException: Unable to find org.apache.hadoop.hbase.ipc.controller.ClientRpcControllerFactory
        at org.apache.hadoop.hbase.util.ReflectionUtils.instantiateWithCustomCtor(ReflectionUtils.java:36)
        at org.apache.hadoop.hbase.ipc.RpcControllerFactory.instantiate(RpcControllerFactory.java:58)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:685)
        ... 35 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.ipc.controller.ClientRpcControllerFactory
 
提交语句已经使用 --jars /opt/hadoopclient/HBase/hbase/lib/phoenix-core-4.4.0-HBase-1.0.jar
不知道为什么还是找不到这个类。
已邀请:

jinqian

赞同来自:

hbase-protcol.java这个包存在冲突。 

要回复问题请先登录注册


中国HBase技术社区微信公众号:
hbasegroup

欢迎加入HBase生态+Spark社区钉钉大群