Questions tagged [hortonworks-sandbox]

1

votes
1

answer
303

Views

Error on check-env.sh installing Apache Kylin on Hortonworks

I'm trying to install Apache Kylin on a Hortonworks Sandbox following the instructions provided on Apache Kylin install. I set up on my .bashrc export KYLIN_HOME='/root/kylin' (inside this folder there are the Kylin Binaries ). In step 3 it say to run bin/check-env.sh to check for a enviroment issu...
Nadia Bastidas
1

votes
1

answer
1.2k

Views

Can not connect to ZooKeeper/Hive from host to Sandbox Hortonworks HDP VM

I downloaded HDP-Sandbox (in an Oracle VirtualBox VM) a while ago, never used it much, and I’m now trying to access data from the outside world using Hive HDBC. I use hive-jdbc 1.2.2 from apache, which I got from mvnrepository, with all the dependencies in the classpath, or hortonworks JDBC got fr...
Sxilderik
1

votes
0

answer
46

Views

hortonworks hbase not accessible

I am using centos-release-6-9.el6.12.3.x86_64 in hortonworks sandbox. I have started hadoop and hbase from ambari server. HMaster and HRegion server started but hquorumpeer is not started. I am able do all operations correctly on hadoop but while creating table in hbase i am getting following except...
rutuja
1

votes
0

answer
66

Views

Error while using Pywebhdfs in Hortonworks

I first opened the rest server on port 8000 using /usr/hdp/current/hbase-master/bin/hbase-daemon.sh start rest -p 8000 --infoport 8001 Then wrote this code in python: from pywebhdfs.webhdfs import PyWebHdfsClient hdfs = PyWebHdfsClient(host='192.168.56.101',port='8000',user_name='maria_dev') my_fil...
Anwesh Mohapatra
1

votes
0

answer
224

Views

how to read data from mongodb in zepplin using spark?

I m working with zeppelin in hdp 2.6 I want to read collection from mongodb using spark2 interpreter util.Properties.versionString spark.version res22: String = version 2.11.8 res23: String = 2.2.0.2.6.4.0-91 I m using MongoDB 3.4.14 mongo-spark-connector 2.2.2 mongo-java-driver 3.5.0 when I try thi...
Chaouki
1

votes
1

answer
186

Views

Unable to connect to Kafka Broker on Hortonworks Sandbox (HDP 2.6) from my Windows Host Machine on Port 6667

I have downloaded Hortonworks Sandbox (IP: 192.179.144.XXX) on my Windows 10 laptop. I have the HDP Sandbox running on VMWare. Below is my sandbox information: Created on: 19_04_2017_19_09_16 for Hadoop stack version: Hadoop 2.7.3.2.6.0.3-8 Ambari Version: 2.5.0.5-1 Ambari Hash: 0b5e975972e7a0b...
Caught At Cover
1

votes
1

answer
30

Views

Hortonworks HDP How to set up a Kerberos enabled Kafka

I have recently downloaded Hortonworks HDP VM. I am able to run Kafka on it. I can produce/consume messages through security-protocol=PLAINTEXT. However, I now want to consume through security-protocol=SASL_PLAINTEXT and Kerberos. I know that I can setup, SASL_PLAINTEXT through Ambari (Screenshot a...
Fawad Shah
1

votes
1

answer
47

Views

Can we perform crud operation on “text file” in hive

I am a newbie to Hadoop, before posting this question I have already searched on google and found all the crud operations example with only ORC files. So wanted to know if we can do the same with a text file or any changes are required in syntax. Thanks in advance.
Nandish B
1

votes
1

answer
158

Views

Talend issue while copying local files to HDFS

Hi I want to know how to copy files to HDFS from source file system(Local File system),if source file already copied to HDFS,then how to eliminate or ignore that file to copy again in HDFS using Talend. Thanks Venkat
venkateswarlu
1

votes
2

answer
1.9k

Views

Sqoop list-databases requires --connection-manager

I'm using Hortonworks Sandbox HDP 2.4 and I can't use the list-databases tool, although import is working. sqoop list-databases --connect jdbc:mysql://:/ --driver com.mysql.jdbc.Driver --username=myusername -P This the result that I get 16/08/11 07:44:47 WARN sqoop.ConnFactory: Parameter --driver i...
menorah84
1

votes
1

answer
288

Views

Using OVA file (VirtualBox image) to create Google Compute Engine VM instance

I want to load the Hortonworks sandbox VirtualBox/VMware image in Google Compute Engine. Is it possible? If yes, how? I am able to load the image locally in the laptop, but that eats up all the resources and slows down everything.
remis haroon
1

votes
1

answer
1.8k

Views

Installing Apache Spark using yum

I am in the process of installing spark in my organization's HDP box. I run yum install spark and it installs Spark 1.4.1. How do I install Spark 2.0? Please help!
Kunal Anand
1

votes
1

answer
583

Views

Importing Appliances (Hortanworks Sandbox) into Virtual box showing up ERROR: Callee RC: NS_ERROR_CALL_FAILED (0x800706BE)

Trying to import Hortonworks Sand box into my virtual box on Mac machine. Upon importing I am getting Callee RC: NS_ERROR_CALL_FAILED (0x800706BE) error message do know how to resolve this
1

votes
1

answer
103

Views

how to ssh-copy id between two instances created on Google Computer Engine

Instance : 1 hostname: hdp-node-1 ip addr: 35.200.132.198 Instance : 2 hostname: hdp-node-2 ip addr: 35.200.208.150 I am able login into both of the instances from my local machine. but i want to set Password-less SSH using public private key pair between these two instances in google cloud platf...
Vinay Maurya
1

votes
1

answer
189

Views

HDFS 'No such file or directory' error when trying to copy csv from one directory to another

I am using Hortonworks Sandbox (not sure if this matters) and was trying to copy a csv file from '/user/root/' directory into '/user/nyc_taxi/' directory but it gave me a 'No such file or directory' error. I am very sure the csv, called 'yellow_tripdata_2016_01.csv', is in the '/user/root/' director...
Stanleyrr
1

votes
1

answer
0

Views

Why do I get “File could only be replicated to 0 nodes” when writing to a partitioned table?

I create an external table in Hive with partitions and then try to populate it from the existing table, however, I get the following exceptions: Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /apps/hive/warehouse/pavel.db...
hey_you
1

votes
1

answer
199

Views

How to ask Ambari service to start automatically

I'm implementing the Ambari service and need to have it started aotumatically after cluster was started. Is there some option in metainfo.xml to enable this?
Akceptor
6

votes
0

answer
634

Views

Can't Import Requests into Python: ImportError no module named urllib3

I've searched on this but other examples I've come across are people having issues installing Requests, my issue is around importing the module: Using Putty (connected to a HDF 2.4 sandbox session) and Python to call an API. Last night, I had this working - installed the Requests module no problem,...
Jon295087
2

votes
1

answer
820

Views

Not able to send json tweets events to Kafka topic/producer using kafka command line

I have created a python script raw_tweets_stream.py to stream twitter data using twitter api. The json data from twitter is pipped to kafka producer using the script below. `python raw_tweets_stream.py | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list localhost:2181 --topic...
gkc123
13

votes
2

answer
9.2k

Views

Got InterruptedException while executing word count mapreduce job

I have installed Cloudera VM version 5.8 on my machine. When I execute word count mapreduce job, it throws below exception. `16/09/06 06:55:49 WARN hdfs.DFSClient: Caught exception java.lang.InterruptedException at java.lang.Object.wait(Native Method) at java.lang.Thread.join(Thread.java:1281) at j...
PUSHPAK GOHEY
7

votes
2

answer
19k

Views

How to load docker image from tar file

I have installed Docker for Windows. I was given the task of working on docker of HortonWorks Docker in windows I don't know. I have downloaded HDP_2.5_docker.tar from here(http://hortonworks.com/downloads/#sandbox) which is 10GB file. How can I load image tar file. I have tried these commands in cm...
User Learning
3

votes
1

answer
314

Views

Import PostgreSQL with Sqoop in Docker

I have a PostgreSQL DB sitting on my local machine (Windows) and I would like to import it into my Hortonworks Sandbox using Apache Sqoop. While something like this sounds great, the complicating factor is that my Sandbox is sitting in a Docker container, so statements such as sqoop list-tables --co...
tgordon18
1

votes
1

answer
1.1k

Views

Execute a java program from nifi with parameters

i want to take some terms from a cassandra database, and then send them as a parameter to java program which will output some data that i will need to save in a cassandra database. Is this possible with nifi, if yes, which processors to use? specially for the execution of the java program, since i'v...
Remis07
2

votes
1

answer
317

Views

Hive query on small dataset never finishes (or OOM)

Performing a simple query on a small sample dataset (195 rows, 22 columns) either throws an out of memory exception, or, following many suggestions to increase memory sizes, never ends. Options tried set hive.optimize.sort.dynamic.partition = true increase tez memory increase memory & decrease shuff...
Laurens Koppenol
2

votes
1

answer
358

Views

how to transfer file from ubuntu desktop to hortonworks sandbox

I have a jar file in local desktop in my ubuntu. Would like to transfer to Horton works sandbox running in vm player .In windows we have winscp to perform it.I don't know how to do it in ubuntu. Could you please help.
user2949241
3

votes
1

answer
743

Views

Apache NiFi - OutOfMemory Error: GC overhead limit exceeded on SplitText processor

I am trying to use NiFi to process large CSV files (potentially billions of records each) using HDF 1.2. I've implemented my flow, and everything is working fine for small files. The problem is that if I try to push the file size to 100MB (1M records) I get a java.lang.OutOfMemoryError: GC overhead...
riccamini
2

votes
2

answer
237

Views

When spark streaming data from Kafka it shows connection timeout exception in Horton works

I want to stream data using spark from Kafka topic in Horton works. I have started the zookeeper and Kafka server. Then I have successfully built a maven project in Eclipse with all the streaming dependencies. Given below is the pom file. POM.XML 4.0.0 test spark-kafka 1.0-SNAPSHOT hortonworks horto...
2

votes
1

answer
3k

Views

Using hive database in spark

I am new in spark and trying to run some queries on tpcds benchmark tables, using HortonWorks Sandbox. http://www.tpc.org/tpcds/ There is no problem while using hive through shell or hive-view on sandbox. The problem is that I don't know how connect to the database if I want to use the spark. How...
vahiiid
2

votes
1

answer
59

Views

Producing from localhost to Kafka in HDP Sandbox 2.6.5 not working

I am writing Kafka client producer as: public class BasicProducerExample { public static void main(String[] args){ Properties props = new Properties(); props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, '127.0.0.1:9092'); props.put(ProducerConfig.ACKS_CONFIG, 'all'); props.put(ProducerConfig.RETRIES...
Aftab Alam
2

votes
0

answer
231

Views

Can't get the location for replica 0 in Phoenix HDP2.6

I enabled Phoenix for HBase on the HDP server. But, if I try to enable sqlline by using the below command: ./sqlline.py localhost:2181:/hbase_unsecure It encounters an error: Error: org.apache.hadoop.hbase.client.RetriesExhaustedException: Can't get the location for replica 0 (state=,code=0) I have...
Tammy
2

votes
1

answer
80

Views

Possibilities for structuring ingested json data using Nifi

Is it possible, using Nifi, to load a json file into a structured table? I've called the following weather forecast data (from 6000 weather stations), which i'm currently loading into HDFS. It all appears on one line: {'SiteRep':{'Wx':{'Param':[{'name':'F','units':'C','$':'Feels Like Temperature...
Jon295087
1

votes
0

answer
280

Views

Unable to determine the current HDP while installing spark client on HDP 2.3

While deploying the hdp cluster setup using Apache Ambari's Automated setup guide, we encountered the following error at the edge node. stderr: 2016-08-08 17:46:03,644 - Could not determine HDP version for component spark-client by calling '/usr/bin/hdp-select status spark-client > /tmp/tmp_vn3OF'...
Prateek.Naik
3

votes
1

answer
1.1k

Views

HiveContext is not reading schema of an Orcfile

When I run the following: val df1 = sqlContext.read.format('orc').load(myPath) df1.columns.map(m => println(m)) The columns are printed as '_col0', '_col1', '_col2' etc. As opposed to their real names such as 'empno', 'name', 'deptno'. When I 'describe mytable' in Hive it prints the column name corr...
DilTeam
0

votes
1

answer
51

Views

how to get started with docker version of hortonworks after install

I followed the Docker installation (using the bash script) to install Hortonworks Sandbox onto a server via the Docker component. It is installed on ubuntu 18.04 server. The install apparently worked. Now what I want to do is be able to get to the website from other machines in the network. My IP...
arcee123
1

votes
0

answer
119

Views

Hbase Error while connecting from Java API

I am getting below error when I execute a JAVA API code to connect Hbase table using below code Java code HBaseConfiguration.addHbaseResources(configuration); LoadIncrementalHFiles loadFfiles = new LoadIncrementalHFiles(configuration); HTable hTable = new HTable(configuration, tableName);...
venkateswarlu