Questions tagged [hbase]

1

votes
3

answer
1.9k

Views

Sending a File is better or a Output Stream

In my project we are saving some files. When a file is requested, I fetch the file from the HBase table and then send it to the browser. Data is saved in Bytes in HBase. So when I fetch the file I am getting it in Bytes. Now I have to convert these Bytes into either File object or in Output Stream....
JHS
1

votes
0

answer
52

Views

HBase : Retention policy for HBase Export

We are using HBase 1.2.3. I am trying to configure HBase backup functionality (Export functionality in 1.2.3 version). Am able to successfully export table on S3. Both full & incremental backups. On S3, all the files goes in default root/base folder and a mapping file (not sure in which language) go...
Sunil Agarwal
1

votes
1

answer
215

Views

Not able to use HBaseTestingUtility with CDH 5.7

I am trying to use HBaseTestingUtility with CDH 5.7 as mentioned in the below blog and github http://blog.cloudera.com/blog/2013/09/how-to-test-hbase-applications-using-popular-tools/ https://github.com/sitaula/HBaseTest I have modified my pom.xml for CDH 5.7 like below 4.0.0 HBaseTest Test 0.0.1-SN...
tuk
1

votes
1

answer
1.3k

Views

Did not find value which can be converted into java.lang.String

I have a JSON input which is to be written into Hbase an example of this is { 'contributors': null, 'truncated': false, 'text': '#Care #MichelleNunn #Stop #Pimping #Haiti #HurricaneMatthew #Give #Answers #Blacklivesmatter https:\/\/t.co\/mmlOawLdCy', 'is_quote_status': false, 'in_reply_to_status_id...
Ram
1

votes
0

answer
166

Views

Insert JSON file into HBase using Hive

I have a simple JSON file that I would like to insert into an HBase table. My JSON file has the following format: { 'word1':{ 'doc_01':4, 'doc_02':7 }, 'word2':{ 'doc_06':1, 'doc_02':3, 'doc_12':8 } } The HBase table is called inverted_index, it has one column family matches. I would like to...
Achraf Oussidi
1

votes
0

answer
68

Views

HBase Versioning: Use built-in system or custom timestamp management?

I am in the process of planning a versioned data store with HBase. In this case 'versioned' means that old data never gets deleted and is retained indefinitly for auditing and analysis purposes. Most reads will still occur on the latest version. HBase itself provides a versioning mechanism, but by d...
Alan47
1

votes
0

answer
415

Views

Cant connect Phoenix to Hbase using sqlline

I am using Hbase 1.2.6, phoenix-4.13.1-HBase-1.2, hadoop 2.9.0, in pseudo distributed setup. I can create tables in hbase shell, and I want to use phoenix to read those tables. I get the following error when running 'sqlline.py localhost:2181:/hbase-unsecure' : Setting property: [incremental, false]...
1

votes
0

answer
46

Views

hortonworks hbase not accessible

I am using centos-release-6-9.el6.12.3.x86_64 in hortonworks sandbox. I have started hadoop and hbase from ambari server. HMaster and HRegion server started but hquorumpeer is not started. I am able do all operations correctly on hadoop but while creating table in hbase i am getting following except...
rutuja
1

votes
0

answer
33

Views

why does hbase need nodemanager when it uses coprocessors

Node Manager is used to start, execute and monitor containers on YARN(containers are assigned to execute map-red jobs). Co-processor on the other hand is a framework which does distributed computation directly within the HBase server processes. I have tables in HBase which I query using phoenix. My...
Aditya
1

votes
0

answer
68

Views

No command builder registered for name extractHBaseCells

I am trying to index data from HBase to Solr and using Hbase-Indexer for the same. But I am getting 'No command builder registered for name: extractHBaseCells' any suggestions. My morphline.conf is morphlines : [{ id : morphline1 importCommands : ['org.kitesdk.morphline.**', 'com.ngdata.**']...
Prayalankar Ashutosh
1

votes
0

answer
30

Views

Difference between major_compact on a table or on all regions of table

Say I have an Hbase table with 10 regions, Which is better approach to run major_compact on the table. To run major_compact on whole table via command: hbase> major_compact 'tableName' or running major_compact on all regions one after another(after major_compact of one region has completed)- hbase>...
Saurabh
1

votes
2

answer
373

Views

HBase Spark connection options [Java]

I am trying to access HBase from a java Spark application running on YARN, but I need some clues on how to do it. I've already searched online, but I cannot find definitive answers. Here they are: The basic one: how do I interact with HBase from Spark? Do I need to set up an HBase connection on each...
riccamini
1

votes
1

answer
522

Views

Hbase data query using rest api

To get data from Hbase table using rest we can use: http://ip:port/tablename/base64_encoded_key My key is byte array of prefix + customer_id + timestamp byte[] rowKey = Bytes.add(Bytes.toBytes(prefix),Bytes.toBytes(customer_id),Bytes.toBytes(timestamp)); My sample key 3\x00\x00\x00\x02I9\xB1\x8B\x00...
kinkajou
1

votes
2

answer
377

Views

How to get column names from column family in HBase using rest api?

I am trying to get column names for a table in hbase using rest api. Url : http://localhost:10005/Customers/schema Am able to get only the column family name ('Info'). But I need to get all the column names present in the table. Please let me know how to get it using rest api.? Thanks in advance.
mRhNs13
1

votes
1

answer
44

Views

How client knows zookeeper is lagging behind in transaction?

Can anyone tell me what happens in following scenario There are five zookeeper servers s1,s2,s3,s4 and s5 When client connected to the s3 it was up-to-date client made a write request to create /test node to s3 it forwarded to leader(s5) As s1,s2,s5 completed that request successfully,Client got the...
1

votes
0

answer
101

Views

Apache Phoenix select with joins execution plan

I have a simple query with two joins that I'm running using apache phoenix driver (v4.13.1): explain select one.pk, two.pk, three.pk from table_one one left join table_two two on two.pk = one.two_id left join table_three three on three.pk = two.three_id . where one.pk = 1 The result looks quite ba...
Art
1

votes
0

answer
404

Views

Getting Regionserver throwing InvalidToken exception in logs

I have noticed following error in my region server logs: org.apache.hadoop.security.token.SecretManager$InvalidToken: access control error while attempting to set up short-circuit access to /apps/hbase/data/data/default/my-table/eb512b4b9f9fa9cb2a1a3930d9c9f18b/r/df1694a4542f419992f86b219541fb6fBloc...
Saurabh
1

votes
1

answer
68

Views

Sync database extraction with Hadoop

Lets say you have periodic task that extract data from a database and loads that data into Hadoop. How does Apache Sqoop/Nifi mantain database sync between the source database (SQL or NoSQL) with destination storage(Hadoop HDFS or HBASE, even S3)? For example, lets say that at time A the database h...
Diego Gallegos
1

votes
0

answer
57

Views

Understanding HBase scan metrics

I am trying to understand the meaning of few of the metrics from HBase ScanMetrics Scan metrics: {count of regions: 1, remote rpc calls: 15, remote rpc retries: 0, Sum of Milli seconds between nexts: 8380} I have set client side caching of rows to 10000 so that each call to next will give me those m...
pc70
1

votes
0

answer
171

Views

phoenix Join tables not using index hint

When using phoenix secondary index, and query non-primary, non-index, non-covered field, hint not working when join tables. Here's the index CREATE INDEX my_index ON table_1 (f1, f2); And this works with index: select /*+ INDEX(table_1 my_index) */ f1, f2, f3 from table_1; This not working with ind...
pinkdawn
1

votes
0

answer
166

Views

Storing small size big quantities image on hdfs for later processing

I am working on a project at which we have a billion of images with their metadata on MongoDB. I want to store this image on HDFS for later image processing. The size of image is between 500K to 4MB, thus, I have the problem of small files with Hadoop. I found 3 main possible solutions for this pro...
bob-cac
1

votes
0

answer
47

Views

Phoenix on Hbase wrong results when region server is down

I have a 4 node cluster. When one of the node is down i see that HBase is running some compactions. During this time if i execute Apache phoenix queries it is giving me wrong results. Once the compactions are completed it is giving me correct results. My replication factor is 3. I am using HDP 2.6....
dsr301
1

votes
0

answer
68

Views

springboot + HBase + Redis junit test case

I have a project with springboot + Hbase + Redis. I want to write some junit test cases from a controller to a service to a HBaseDao, a whole flow operation. Because the project are committed by different teams in three cities, and every team has its env, so I cannot hardcode Hbase rootDir, zookeep...
Liu guanghua
1

votes
0

answer
318

Views

Java Heap Space out of memory on HBase Table scan

In my application, I am scanning a table in HBase to fetch the data to be shown on UI. Data is huge in amount and code is giving the below error. Provided Server Configuration is: server -Xms2048m -Xmx4096m 'Failure in caller transaction.: java.lang.OutOfMemoryError: Java heap space' Code: HTable...
whywake
1

votes
1

answer
429

Views

Modify existing table split in hbase

I have a HBase table created with 20 pre-created splits based on row key prefix. Now I want to increase the number of splits to 50 using the same method. Is there any way to do that in HBase? Dropping and re-creating the table with new splits is not an option as I have some hundreds of millions of r...
KNP
1

votes
0

answer
39

Views

HBase - SAS Integration and Read

I have cloudera cluster(kerberos enabled) and HBase is running in it. I need few tables in HBase with the filter condition which needs to read/write from external SAS server. I am trying to achieve this through Thrift and Python whereas I have installed python in my SAS Server and accessing HBase t...
Srini Ravi
0

votes
0

answer
5

Views

Hbase coporcessor connot be loaded the new one?

I have created a coprocessor and uploaded it to hdfs. I loaded this coprocessor with alter command for my table, alter 'my:table', METHOD => 'table_att', 'coprocessor$1' => 'hdfs://master/user/coprocessor-1.0.jar|com.xxx.IndexCoprocessor||' But, I modified my program and re-uploaded it to hdfs after...
another JJz
1

votes
0

answer
33

Views

JanusGraph Hbase repeatedly calls HBaseKeyColumnValueStore.getFilters()

We are running a Java application, which uses a JanusGraph backed by a Hbase table on a Cloudera cluster. We use the janusgraph-hbase dependency, v0.2. When running our app, we see these lines appear in the logs: 20180330 15:00:27;DEBUG;HBaseKeyColumnValueStore:145;Generated HBase Filter FilterList...
Charles
1

votes
0

answer
95

Views

Microsoft Power BI free connector to HBase

Is there a free way of connecting Power BI to HBase? I can connect using ODBC/ADO.Net HBase driver which is offered by CData Software but I was wondering if it's possible to connect using any free driver. May be it's possible to use some framework or SDK for this goal in PBI?
Ilya Urikh
1

votes
0

answer
133

Views

Apache Nifi HBASE lookup

I am new on Apache Nifi We create Nifi flow, which is consuming json data from kafka and the results are being sent to another kafka topic after enrichment. However HBase lookup does not return value of key. Instead it returns key, value pair like MapRecord[{SERIAL_NUM=123456789}]. But I require on...
erkan.oktay
1

votes
0

answer
257

Views

How concatenate column varchar values in group by?

I want to do a select where for one column, I concatenate the values. For example, if I have the rows: ID NAME Friend 1 Joe Fred 2 Jeff Fred 3 Joe Jack 4 Joe Sally And I grouped by name, I would get: Joe Fred,Jack,Sally Jeff Fred I...
Don Rhummy
1

votes
0

answer
57

Views

Node.JS - node-hbase Scan not working for more than 1000 records

I want to get records from an HBase table into Node.JS code using node-hbase module which connects to the rest server. But am unable to get the next set of records after the first batch. So my final count is 1000(which is the batch size). But the actual size is more than 7000. var hbase = require('h...
Pushkar Nagpal
1

votes
0

answer
81

Views

Hbase Cluster down due to the one of the Hadoop HA namenode went unreachable

I have a HA Hadoop with active and standby namenode and a HBase setup with master and backup master. Active nameode and Hbase Master runs in the same Physical machine. In the same way, standby namenode and backup hbase master in another machine. Hadoop version - 2.7.3 Hbase version - 1.2.6 The acti...
Roshan
1

votes
2

answer
140

Views

Convert String to Ruby byte array when using Hbase Shell

I want to convert a String to a Ruby byte array. I have a String and saving it in Hbase as binary byte array. Need to scan with some filter on key which is a binary byte array. Have a String like 'U92w8GSBJ2BiHzGg' and need its representation like '\xFF\xA4\xDD\x13\x003\xE4\x85\xC7\x9D\xD5ZY\xF0\x1...
minhas23
1

votes
0

answer
236

Views

hbase rowkey filter on multiple values

I have created an Hbase table rowkey with combination of multiple column values. my data of rowkey in hbase look like below. '123456~raja~ram~45000~mgr~20170116' '123456~raghu~rajan~65000~mgr~20150106' i am trying to filter condition just like in sql as select * from table where deptid =123456 and...
sri
1

votes
1

answer
246

Views

HBase Concurrent / Parallel Scan from Spark 1.6, Scala 2.10.6 besides multithreading

I have a list of rowPrefixes Array('a', 'b', ...) I need to query HBase (using Nerdammer) for each of the rowPrefix. My current solution is case class Data(x: String) val rowPrefixes = Array('a', 'b', 'c') rowPrefixes.par .map( rowPrefix => { val rdd = sc.hbaseTable[Data]('tableName') .inColumnFamil...
user9395367
1

votes
0

answer
97

Views

Unable to connect to HBase table using Java

I am trying to connect to Hbase (version: 1.2.0) deployed on a different machine. I am using Eclipse and below is the code that I have written: HTable table = null; Configuration configuration = HBaseConfiguration.create(); configuration.set('hbase.zookeeper.quorum', '192.168.0.191'); configuration....
whywake
1

votes
0

answer
39

Views

Getting error connecting for the first time to HBase using Phoenix driver

Using Phoenix driver I am connecting to HBase. Everything works, but there is an error stacktrace in the log while getting a connection. Again, it works fine, functionally no issues. DataSource dataSource = getDataSource(); TransactionFactory transactionFactory = new JdbcTransactionFactory(); Envir...
surya
1

votes
0

answer
118

Views

delete hbase on spark rdd,Task not serializable

i want to delete data from hbase with rdd, here are my codes def delGraphIdVertexInfo(vertexInfoRDD: RDD[(BqsVertex, String)]): Unit = { vertexInfoRDD.foreachPartition(partition => { val hc = HBaseConfiguration.create() val cn = ConnectionFactory.createConnection(hc) val userTable = TableName.valueO...
user7687835
1

votes
1

answer
1.1k

Views

hbase shell cannot use: ERROR: KeeperErrorCode = NoNode for /hbase/master

I try to install hbase in pseudo-distributed mode. Due to this question ,I cannot use hbase now. Every code in the hbase shell cannot run and all of them get this error tip : ERROR: KeeperErrorCode = NoNode for /hbase/master In my ubuntu 17 I installed hadoop ,i'm sure that my hdfs location match my...
Solodye

View additional questions