Can not connect to ZooKeeper/Hive from host to Sandbox Hortonworks HDP VM

Refresh

April 2019

Views

1.2k time

1

I downloaded HDP-Sandbox (in an Oracle VirtualBox VM) a while ago, never used it much, and I’m now trying to access data from the outside world using Hive HDBC.

I use hive-jdbc 1.2.2 from apache, which I got from mvnrepository, with all the dependencies in the classpath, or hortonworks JDBC got from their repository

http://mvnrepository.com/artifact/org.apache.hive/hive-jdbc/1.2.2

or

http://repo.hortonworks.com/content/repositories/releases/org/apache/hive/hive-jdbc/1.2.1000.2.6.4.0-91

I wrote this very basic program

import java.sql.Connection;
import java.sql.DriverManager;

public class Main {

    public static void main(final String[] args) {

        try {
            final Class<?> driver = Class.forName("org.apache.hive.jdbc.HiveDriver");
            DriverManager.setLoginTimeout(10);
            try (final Connection connection = DriverManager.getConnection("jdbc:hive2://sandbox.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2")) {
                //
            }
        } catch (final Exception e) {
            e.printStackTrace();
        }
    }
}

Connection fails with a Null Pointer Exception or a timeout (see the output logs below, one for each JDBC driver I used), I guess it’s caused by a configuration error – even though I use my sandbox out of the… box.

My sandbox version

Sandbox information:
Created on: 25_10_2016_08_11_26 for
Hadoop stack version:  Hadoop 2.7.3.2.5.0.0-1245
Ambari Version: 2.4.0.0-1225
Ambari Hash: 59175b7aa1ddb74b85551c632e3ce42fed8f0c85
Ambari build:  Release : 1225
Java version:  1.8.0_111
OS Version:  CentOS release 6.8 (Final)

My VM screen

HDP 2.5
http://hortonworks.com
To initiate your Hortonworks Sandbox session,
please open a browser and enter this address
in the browser's address field:
http://127.0.0.1:8888/

My hosts file :

…/…
127.0.0.1 sandbox.hortonworks.com
…/…

Hive Summary from the Ambari screen

Hive Metastore        Started No alerts
HiveServer2           Started No alerts
WebHCat Server        Started No alerts
Hive Client           1 Hive Client Installed
HiveServer2 JDBC URL  jdbc:hive2://sandbox.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2

Zookeeper Summary

ZooKeeper Server    Started No alerts
ZooKeeper Client    1 ZooKeeper Client Installed

I’m stuck, any help would be greatly appreciated!

Thanks !


Program outputs:

With apache hive jdbc 1.2.2:

16:52:06,378  INFO hive.jdbc.Utils: Supplied authorities: sandbox.hortonworks.com:2181
16:52:06,500  INFO curator.framework.imps.CuratorFrameworkImpl: Starting
16:52:06,503 DEBUG curator.CuratorZookeeperClient: Starting
16:52:06,503 DEBUG curator.ConnectionState: Starting
16:52:06,503 DEBUG curator.ConnectionState: reset
16:52:06,515  INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
16:52:06,515  INFO zookeeper.ZooKeeper: Client environment:java.version=9.0.1
16:52:06,515  INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
16:52:06,515  INFO zookeeper.ZooKeeper: Client environment:java.home=D:\JDK\jdk9
16:52:06,516  INFO zookeeper.ZooKeeper: Initiating client connection, connectString=sandbox.hortonworks.com:2181 sessionTimeout=60000 [email protected]
16:52:06,521 DEBUG zookeeper.ClientCnxn: zookeeper.disableAutoWatchReset is false
16:52:06,631  INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
16:52:06,633  INFO zookeeper.ClientCnxn: Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session
16:52:06,635 DEBUG zookeeper.ClientCnxn: Session establishment request sent on 127.0.0.1/127.0.0.1:2181
16:52:06,645  INFO zookeeper.ClientCnxn: Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x160ff8bf98b0008, negotiated timeout = 40000
16:52:06,650  INFO curator.framework.state.ConnectionStateManager: State change: CONNECTED
16:52:06,663 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0008, packet:: clientPath:null serverPath:null finished:false header:: 1,12  replyHeader:: 1,1347,0  request:: '/hiveserver2,F  response:: v{'serverUri=sandbox.hortonworks.com:10000;version=1.2.1000.2.5.0.0-1245;sequence=0000000007},s{467,467,1477381899524,1477381899524,0,15,0,0,10,1,1340} 
16:52:06,669 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0008, packet:: clientPath:null serverPath:null finished:false header:: 2,4  replyHeader:: 2,1347,0  request:: '/hiveserver2/serverUri=sandbox.hortonworks.com:10000;version=1.2.1000.2.5.0.0-1245;sequence=0000000007,F  response:: #686976652e736572766572322e61757468656e7469636174696f6e3d4e4f4e453b686976652e736572766572322e7472616e73706f72742e6d6f64653d62696e6172793b686976652e736572766572322e7468726966742e7361736c2e716f703d617574683b686976652e736572766572322e7468726966742e62696e642e686f73743d73616e64626f782e686f72746f6e776f726b732e636f6d3b686976652e736572766572322e7468726966742e706f72743d31303030303b686976652e736572766572322e7573652e53534c3d66616c7365,s{1340,1340,1516116649382,1516116649382,0,0,0,99360168454324228,213,0,1340} 
16:52:06,670  INFO hive.jdbc.ZooKeeperHiveClientHelper: Selected HiveServer2 instance with uri: hive.server2.authentication=NONE;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=sandbox.hortonworks.com;hive.server2.thrift.port=10000;hive.server2.use.SSL=false
16:52:06,670 DEBUG curator.framework.imps.CuratorFrameworkImpl: Closing
16:52:06,671 DEBUG curator.CuratorZookeeperClient: Closing
16:52:06,671 DEBUG curator.ConnectionState: Closing
16:52:06,672 DEBUG zookeeper.ZooKeeper: Closing session: 0x160ff8bf98b0008
16:52:06,672 DEBUG zookeeper.ClientCnxn: Closing client for session: 0x160ff8bf98b0008
16:52:06,674 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0008, packet:: clientPath:null serverPath:null finished:false header:: 3,-11  replyHeader:: 3,1348,0  request:: null response:: null
16:52:06,674 DEBUG zookeeper.ClientCnxn: Disconnecting client for session: 0x160ff8bf98b0008
16:52:06,674  INFO zookeeper.ZooKeeper: Session: 0x160ff8bf98b0008 closed
16:52:06,674  INFO hive.jdbc.Utils: Resolved authority: hive.server2.authentication=NONE;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=sandbox.hortonworks.com;hive.server2.thrift.port=10000;hive.server2.use.SSL=false
16:52:06,675  INFO zookeeper.ClientCnxn: EventThread shut down
16:52:06,731  INFO hive.jdbc.HiveConnection: Will try to open client transport with JDBC Uri: jdbc:hive2://hive.server2.authentication=NONE;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=sandbox.hortonworks.com;hive.server2.thrift.port=10000;hive.server2.use.SSL=false/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
16:52:06,733 DEBUG thrift.transport.TSaslTransport: opening transport [email protected]
java.lang.NullPointerException
    at org.apache.thrift.transport.TSocket.open(TSocket.java:170)
    at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:266)
    at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
    at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:206)
    at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:178)
    at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
    at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:678)
    at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:252)
    at com.mycompany.hadoop.hiveconnect.Main.main(Main.java:13)

With Hortonworks hive jdbc 1.2.1000.2.6.4.0-91

16:49:57,414  INFO hive.jdbc.Utils: Supplied authorities: sandbox.hortonworks.com:2181
16:49:57,544  INFO curator.framework.imps.CuratorFrameworkImpl: Starting
16:49:57,546 DEBUG curator.CuratorZookeeperClient: Starting
16:49:57,546 DEBUG curator.ConnectionState: Starting
16:49:57,546 DEBUG curator.ConnectionState: reset
16:49:57,555  INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.6-91--1, built on 01/04/2018 10:14 GMT
16:49:57,555  INFO zookeeper.ZooKeeper: Client environment:java.version=9.0.1
16:49:57,555  INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
16:49:57,555  INFO zookeeper.ZooKeeper: Client environment:java.home=D:\JDK\jdk9
16:49:57,556  INFO zookeeper.ZooKeeper: Initiating client connection, connectString=sandbox.hortonworks.com:2181 sessionTimeout=60000 [email protected]
16:49:57,560 DEBUG zookeeper.ClientCnxn: zookeeper.disableAutoWatchReset is false
16:49:57,668  INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
16:49:57,673  INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /127.0.0.1:17993, server: 127.0.0.1/127.0.0.1:2181
16:49:57,674 DEBUG zookeeper.ClientCnxn: Session establishment request sent on 127.0.0.1/127.0.0.1:2181
16:49:57,787  INFO zookeeper.ClientCnxn: Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x160ff8bf98b0006, negotiated timeout = 40000
16:49:57,807  INFO curator.framework.state.ConnectionStateManager: State change: CONNECTED
16:49:58,378 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0006, packet:: clientPath:null serverPath:null finished:false header:: 1,12  replyHeader:: 1,1343,0  request:: '/hiveserver2,F  response:: v{'serverUri=sandbox.hortonworks.com:10000;version=1.2.1000.2.5.0.0-1245;sequence=0000000007},s{467,467,1477381899524,1477381899524,0,15,0,0,10,1,1340} 
16:49:58,400 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0006, packet:: clientPath:null serverPath:null finished:false header:: 2,4  replyHeader:: 2,1343,0  request:: '/hiveserver2/serverUri=sandbox.hortonworks.com:10000;version=1.2.1000.2.5.0.0-1245;sequence=0000000007,F  response:: #686976652e736572766572322e61757468656e7469636174696f6e3d4e4f4e453b686976652e736572766572322e7472616e73706f72742e6d6f64653d62696e6172793b686976652e736572766572322e7468726966742e7361736c2e716f703d617574683b686976652e736572766572322e7468726966742e62696e642e686f73743d73616e64626f782e686f72746f6e776f726b732e636f6d3b686976652e736572766572322e7468726966742e706f72743d31303030303b686976652e736572766572322e7573652e53534c3d66616c7365,s{1340,1340,1516116649382,1516116649382,0,0,0,99360168454324228,213,0,1340} 
16:49:58,402 DEBUG curator.framework.imps.CuratorFrameworkImpl: Closing
16:49:58,403 DEBUG curator.CuratorZookeeperClient: Closing
16:49:58,403 DEBUG curator.ConnectionState: Closing
16:49:58,405 DEBUG zookeeper.ZooKeeper: Closing session: 0x160ff8bf98b0006
16:49:58,405 DEBUG zookeeper.ClientCnxn: Closing client for session: 0x160ff8bf98b0006
16:49:58,466 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0006, packet:: clientPath:null serverPath:null finished:false header:: 3,-11  replyHeader:: 3,1344,0  request:: null response:: null
16:49:58,466 DEBUG zookeeper.ClientCnxn: Disconnecting client for session: 0x160ff8bf98b0006
16:49:58,466  INFO zookeeper.ZooKeeper: Session: 0x160ff8bf98b0006 closed
16:49:58,467  INFO hive.jdbc.Utils: Resolved authority: sandbox.hortonworks.com:10000
16:49:58,467  INFO zookeeper.ClientCnxn: EventThread shut down
16:49:58,545 DEBUG thrift.transport.TSaslTransport: opening transport [email protected]
16:49:58,549 DEBUG thrift.transport.TSaslClientTransport: Sending mechanism name PLAIN and initial response of length 20
16:49:58,550 DEBUG thrift.transport.TSaslTransport: CLIENT: Writing message with status START and payload length 5
16:49:58,551 DEBUG thrift.transport.TSaslTransport: CLIENT: Writing message with status COMPLETE and payload length 20
16:49:58,551 DEBUG thrift.transport.TSaslTransport: CLIENT: Start message handled
16:49:58,551 DEBUG thrift.transport.TSaslTransport: CLIENT: Main negotiation loop complete
16:49:58,551 DEBUG thrift.transport.TSaslTransport: CLIENT: SASL Client receiving last message
16:50:08,552  WARN hive.jdbc.HiveConnection: Failed to connect to sandbox.hortonworks.com:10000
16:50:08,553  INFO curator.framework.imps.CuratorFrameworkImpl: Starting
16:50:08,554 DEBUG curator.CuratorZookeeperClient: Starting
16:50:08,554 DEBUG curator.ConnectionState: Starting
16:50:08,554 DEBUG curator.ConnectionState: reset
16:50:08,554  INFO zookeeper.ZooKeeper: Initiating client connection, connectString=sandbox.hortonworks.com:2181 sessionTimeout=60000 [email protected]
16:50:08,557  INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
16:50:08,558  INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /127.0.0.1:18018, server: 127.0.0.1/127.0.0.1:2181
16:50:08,558 DEBUG zookeeper.ClientCnxn: Session establishment request sent on 127.0.0.1/127.0.0.1:2181
16:50:08,592  INFO zookeeper.ClientCnxn: Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x160ff8bf98b0007, negotiated timeout = 40000
16:50:08,592  INFO curator.framework.state.ConnectionStateManager: State change: CONNECTED
16:50:08,594 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0007, packet:: clientPath:null serverPath:null finished:false header:: 1,12  replyHeader:: 1,1345,0  request:: '/hiveserver2,F  response:: v{'serverUri=sandbox.hortonworks.com:10000;version=1.2.1000.2.5.0.0-1245;sequence=0000000007},s{467,467,1477381899524,1477381899524,0,15,0,0,10,1,1340} 
16:50:08,594 DEBUG curator.framework.imps.CuratorFrameworkImpl: Closing
16:50:08,594 DEBUG curator.CuratorZookeeperClient: Closing
16:50:08,594 DEBUG curator.ConnectionState: Closing
16:50:08,594 DEBUG zookeeper.ZooKeeper: Closing session: 0x160ff8bf98b0007
16:50:08,594 DEBUG zookeeper.ClientCnxn: Closing client for session: 0x160ff8bf98b0007
16:50:08,605 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0007, packet:: clientPath:null serverPath:null finished:false header:: 2,-11  replyHeader:: 2,1346,0  request:: null response:: null
16:50:08,605 DEBUG zookeeper.ClientCnxn: Disconnecting client for session: 0x160ff8bf98b0007
16:50:08,605  INFO zookeeper.ZooKeeper: Session: 0x160ff8bf98b0007 closed
16:50:08,605  INFO zookeeper.ClientCnxn: EventThread shut down
16:50:08,605 ERROR hive.jdbc.Utils: Unable to read HiveServer2 configs from ZooKeeper
java.sql.SQLException: Could not open client transport for any of the Server URI's in ZooKeeper: java.net.SocketTimeoutException: Read timed out
    at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:228)
    at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:166)
    at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
    at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:678)
    at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:252)
    at com.mycompany.hadoop.hiveconnect.Main.main(Main.java:13)
Caused by: org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out
    at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
    at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
    at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:178)
    at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:307)
    at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
    at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:204)
    ... 5 more
Caused by: java.net.SocketTimeoutException: Read timed out
    at java.base/java.net.SocketInputStream.socketRead0(Native Method)
    at java.base/java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
    at java.base/java.net.SocketInputStream.read(SocketInputStream.java:171)
    at java.base/java.net.SocketInputStream.read(SocketInputStream.java:141)
    at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
    at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
    at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:345)
    at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
    ... 10 more

1 answers

0

Well, I could not solve this.

But later on, I downloaded the 2.6.4 sandbox and, magic, it worked like a charm!