Questions tagged [apache-zookeeper]

1

votes
2

answer
430

Views

Differences between zookeeper-server-start.sh and kafka-server-start.sh

Is one of them more recommended/preferred to use than the other?
Danny
1

votes
2

answer
1.4k

Views

how to run SolrCloud with docker-compose

Please help me with docker-compose file. Right now, i'm using Solr in docker file, but i need to change it to SolrCloud. I need 2 Solr instances, an internal Zookeeper and docker (local). This is an example of docker-compose file I did: version: '3' services: mongo: image: mongo:latest container_...
Ronny Lee
1

votes
1

answer
620

Views

Solrcloud - remove a node

I have a Solrcloud setup, which runs 3 ZKs and 3 Solrs (version 4.10.3). I would like to take out one of the Solr servers completely from this setup so that I only have 2 Solrs and 3 ZKs. I've tried googling and the only results I can find are to remove replicas or collections but not to remove a n...
Gabbar
1

votes
1

answer
640

Views

Cannot delete a Kafka topic in Windows

I have set the Zookeeper properties of 'delete.topic.enable' to true. But I still cannot delete the topic. When I do mvn install or mvn test, I got following problems: WARN Error processing kafka.log:type=LogManager,name=LogDirectoryOffline,logDirectory=C:\Users\extznq\AppData\Local\Temp\EH4Test7267...
Spider
1

votes
0

answer
246

Views

What parts of Hyperledger Fabric effect the scalability?

So from my understanding, kafka based ordering is what would be used in a production environment. Within a Kafka based ordering environment we have: Zookeeper nodes Kafka nodes Orderer Nodes Peer Nodes Within this environment which of these would effect scalability most or at all? I am currently w...
zonked
1

votes
0

answer
666

Views

Group consumer cannot consume messages if kafka service on specific node in test cluster is down

I have three servers: blade1 (192.168.112.31), blade2 (192.168.112.32) and blade3 (192.168.112.33). On each of servers kafka_2.11-1.0.0 is installed. On blade 3 (192.168.112.33:2181) zookeeper is installed as well. I have created a topic repl3part5 with the following line: bin/kafka-topics.sh --zo...
user3225309
1

votes
1

answer
23

Views

The command status stop “Creating job WordCountTopology” after submit a Topology

I attempted to build a Heron Cluster using Apache Mesos, Apache Aurora, ZooKeeper and HDFS. However, When I submit the WordCountTopology after finished it, the command ouput as following: Stopping the 'Creating job WordCountTopology'. [email protected]:~/.heron/conf/aurora$ heron submit aurora/yitian/de...
Yitian Zhang
1

votes
1

answer
631

Views

Failed to start broker after enabling Zookeeper Authentication

Started Zookeeper with the following properties ie zookeeper.properties dataDir=/tmp/zookeepeeer clientPort=2186 maxClientCnxns=0 auto.offset.reset=smallest authProvider.1=org.apache.zookeeper.server.auth.DigestAuthenticationProvider jaasLoginRenew=36000...
jewel
1

votes
1

answer
44

Views

How client knows zookeeper is lagging behind in transaction?

Can anyone tell me what happens in following scenario There are five zookeeper servers s1,s2,s3,s4 and s5 When client connected to the s3 it was up-to-date client made a write request to create /test node to s3 it forwarded to leader(s5) As s1,s2,s5 completed that request successfully,Client got the...
1

votes
1

answer
1k

Views

Confluent Start -> Schema Registry Failed to Start

When I start Confluent, Schema-registry fails, preventing the process from completing successfully. This is the response I get: Starting zookeeper zookeeper is [UP] Starting kafka kafka is [UP] Starting schema-registry Schema Registry failed to start schema-registry is [DOWN] Starting kafka-rest Kaf...
Scicrazed
1

votes
0

answer
139

Views

Zookeper server does not try to establish quorum

I downloaded Zookeeper and unpacked in 3 directories: z1, z2 and z3. In z1, under /conf/zoo.cfg I put this configuration: tickTime=2000 initLimit=10 syncLimit=5 dataDir=./data clientPort=2181 server.1=127.0.0.1:2222:2223 server.2=127.0.0.1:3333:3334 server.3=127.0.0.1:4444:4445 In z2 and z3 I put th...
octavian
1

votes
0

answer
65

Views

Good way to get streaming Twitter data using Apache Storm?

I'm building a Twitter crawler system. Requirement is to crawl both Twitter Profile and Twitter Streaming. There's a project manager which puts all projects (1 project is a Twitter profile, or a keyword for Twitter Streaming) into Kafka. Then Storm will read from Kafka to get project metadata and st...
keduadoi
1

votes
0

answer
277

Views

Specified config does not exist in ZooKeeper: solr

I have installed solrcloud with zookeeper 3 nodes. I have 5 collections in solr each having different config_name I have stopped solr & zookeeper instances & start it again but when I started the instances it gives below error - 'collection1_shard1_replica2: org.apache.solr.common.cloud.ZooKeeperE...
shailesh
1

votes
0

answer
453

Views

Confluent's Kafka and Zookeeper Docker Images don't play well on Mac OS X

Trying to run the Docker Images for Kafka & Zookeeper from confluentinc on my Mac. Unfortunately, the commands that the Confluent Docker Quickstart documentation provides utilize net=host which does not work on Mac.. Here are the original docker start-up scripts: Zookeeper: docker run -d \ --net=ho...
Jared Hooper
1

votes
2

answer
298

Views

Kafka cluster loses messages after zookeeper restart

I'm starting a cluster of kafka brokers using Docker (5 brokers for example, one broker per container). Kafka version 2.12-0.11.0.0, Zookeeper 3.4.10. The scenario: Starting 1st broker with config below zoo.cfg tickTime=2000 initLimit=10 syncLimit=5 dataDir=/opt/zookeeper/data clientPort=2181 maxCli...
usbulat
1

votes
0

answer
173

Views

Zookeeper for High availability

How does zookeeper work in the following situation. Consider I am having 3 (1,2,3) vm's and different services are running at their endpoints. My entire administration setup (TAC) is available only on the 1st vm (virtual machine) that means whenever a client wants to connect, it would by default...
many
1

votes
0

answer
37

Views

Upload file(binary) into Zookeeper (SolrCloud)

I am using SolrCloud on Windows and I need to upload OpenNLP model files(binary files) into Zookeeper. How can I do this? Here are my related questions: Can not apply patch LUCENE-2899.patch to SOLR on Windows SolrCloud OpenNLP error Can't find resource 'opennlp/en-sent.bin' in classpath or '/config...
1

votes
1

answer
438

Views

Root cause for Connection broken for id 1, my id = 3, error =

I am using Confluent 4 for kafka and zookeeper installation. On our Kafka Cluster environment (of 3 brokers and 3 zookeeper nodes running on 3 aws instances) we are seeing a set of below warnings, repeatedly getting recorded in the broker's server.log file. We have not observed any functionality is...
1

votes
0

answer
103

Views

Zookeeper unable to talk to new Kafka broker

In an attempt to reduce the storage on my AWS instance I decided to launch a new, smaller instance and setup Kafka again from scratch using the Ansible playbook we had from before. I then terminated the old, larger instance and took its IP address that it and the other brokers were using and put it...
Ryan Grush
1

votes
1

answer
699

Views

Find node in Zookeeper

I want to know if there is a way to walk the Zookeeper's (ZK) in memory database and find if any particular node exists. Something similar to find . -name file inside ZK I am logged in to ZK using zkCli.
Vikas Goel
1

votes
0

answer
129

Views

takeLeadership() method of LeaderSelectorListener not being called

I have a standalone zookeeper server running. client = CuratorFrameworkFactory.newClient(zkHostPorts, retryPolicy); client.start(); assertThat(client.checkExists().forPath('/')).isNotNull(); // working listener = new LeaderSelectorListenerAdapter() { @Override public void takeLeadership(CuratorFrame...
Naveen Kumar
1

votes
1

answer
1.1k

Views

hbase shell cannot use: ERROR: KeeperErrorCode = NoNode for /hbase/master

I try to install hbase in pseudo-distributed mode. Due to this question ,I cannot use hbase now. Every code in the hbase shell cannot run and all of them get this error tip : ERROR: KeeperErrorCode = NoNode for /hbase/master In my ubuntu 17 I installed hadoop ,i'm sure that my hdfs location match my...
Solodye
1

votes
0

answer
81

Views

Unable to connect zookeeper node externally

I have a DCOS cluster spun up using the cloudformation template here: https://downloads.dcos.io/dcos/stable/aws.html I'm able to connect remotely to the default dcos zookeeper after the default installation telnet 2181 however when I install the kafka-zookeeper service, I'm unable to connect to tho...
Mars01
1

votes
2

answer
270

Views

Set zookeeper host and port for spring-cloud-bus

I have a project with spring-cloud-starter-bus-kafka and I set the kafka URL inside application.yml like so: spring.kafka.bootstrap-servers=localhost:9092 This works find when kafka and zookeeper are deployed locally, but if I move kafka and zookeeper to their own servers I get an error when spring-...
madmaux
1

votes
1

answer
266

Views

Difference between Zookeeper and a managed replicated database service

I just came across Zookeeper and am wondering as to what's the difference between Zookeeper and an available, consistent, durable, distributed, replicated database service like AWS DynamoDB or even AWS S3(storage service) for that matter. The key features like configuration management, distributed s...
Kumar Mrinal
1

votes
0

answer
229

Views

Zookeeper server health check/monitor using python kazoo client

I have a python script to monitor the Zookeeper server health check - basically check if Zookeeper is up and running. This runs every minute. So I have just written a simple python method which will create a new zookeeper kazoo client and then start and stop the client. def __zookeeper_check(self)...
Xavier DSouza
1

votes
0

answer
231

Views

CuratorFramework spring boot

So if I have 2 services, A and B. Each service implements CuratorFramework, but access the same Zookeeper cluster. Now these are spring boot applications. Once i run these spring boot apps, fire some requests at it, over a period of time my service started throwing OutOfMemory: Unable to create new...
user1510266
1

votes
0

answer
36

Views

Warning when deleting processors using Nifi 1.6

I recently upgraded the test environment to Nifi 1.6 on Centos 7. I am running a three node cluster using the Nifi internal zookeeper version. If I drag in a new processor and then delete it I get the following stack trace in the logs. WARN StandardStateManagerProvider Component with ID {} was...
dmarrow4
1

votes
0

answer
87

Views

Zookeeper folder missing in Solr Cloud admin UI

I have 3 Solr 7.3 and 3 zookeeper 3.4.12 servers. When I look in the cloud tree I can see the configsets from zookeeper but I am missing the zookeeper folder. Is that an indication of an error? I am asking this because I am unable to create a collection without getting a timeout. And I am looking fo...
Martin Andersen
1

votes
0

answer
65

Views

Deleting ZooKeeper path with 100,000+ nodes fails with OutOfMemoryError

I have a ZooKeeper path with hundreds of thousands of nodes (one parent, 100,000+ children, no further children). If I try to delete it using the CLI and rmr command, it waits for about a minute and fails with the following stacktrace: Exception in thread 'main' java.lang.OutOfMemoryError: Java heap...
Alfredo Gimenez
1

votes
1

answer
243

Views

Zookeeper running or not in relation to standard port 2181 usage?

CLOUDERA QUICKSTART 5.13 as follows. I am not sure whether zookeeper out of the box is running or not, and if so, then if it would work reliably? I got this when trying to run zookeeper from within the from kafka supplied version that I downloaded, in standalone mode: [2018-06-17 00:49:32,847] INFO...
thebluephantom
1

votes
0

answer
138

Views

Zookeeper service discovery and Docker

We are migrating from a monolithic application to microservices architectures. Currently we have microservices (mostly spring boot) running on VM, which are used by the front with zookeeper service discovery. I set up a docker swarm, and we'd like to migrate our microservices to docker containers, b...
A.L.
1

votes
1

answer
256

Views

Spark unable to read kafka Topic and gives error “ unable to connect to zookeeper server within timeout 6000”

I'm trying to Execute the Example program given in Spark Directory on HDP cluster '/spark2/examples/src/main/python/streaming/kafka_wordcount.py' which tries to read kafka topic but gives Zookeeper server timeout error. Spark is installed on HDP Cluster and Kafka is running on HDF Cluster, both ar...
Mahesh
1

votes
0

answer
195

Views

ERROR pool dependentstartup event buffer overflowed, discarding event 1980

I am running supervisord. The re-read and restart are done successfully. However, when I run the command: ps -ef | grep python, I am getting only one file running instead of 5. Also, I am getting the error in the status: Jun 25 13:38:18 woodward-testing supervisord[664]: 2018-06-25 13:38:18,308 ERRO...
hasherBaba
1

votes
1

answer
782

Views

org.apache.curator.ConnectionState Connection timed out for connection string

I have been facing this issue for my Solr instance which is managed by Zookeeper. It appears that Zookeeper is able to send requests to Zookeeper which momentarily accepts the request and then refuses it. In Zookeeper logs, I have been seeing this error: INFO org.apache.zookeeper.ZooKeeper.Clien...
Nitin Agrawal
1

votes
0

answer
99

Views

Remove Hyperledger Fabric Network with Kafka orderer

In order to bring down a hyperledger fabric network with Kafka-Zookeeper orderers, what components [docker images/containers etc ] will need to be removed in order, so that the system has no residual data of previous network?
anjy
1

votes
0

answer
274

Views

Kafka-Manager showing Inactive Consumer Groups and incorrect Lag

I have a kafka cluster of 3 separate physical nodes, with each hosting 1 zk and 1 broker each. We are using Kafka 0.10.1.1. The Topics are created on ZK ensemble and the brokers connect to the ZK ensemble. My Producer and Consumer is a spring-boot app on version 1.5.13 RELEASE. We have a few topics...
Divs
1

votes
1

answer
302

Views

How to retrieve details of all broker from kafka cluster

How should we retrieve the full details of all brokers(connected/disconnected) from kafka cluster/zookeeper ? I found following way to fetch only active brokers, but I want the the IP address of broker which is serving previously in cluster but it is disconnected now Following code snippet gives lis...
MAS
1

votes
0

answer
160

Views

Kakfa 0.10.2.1 producer close method issue

We are creating new producer connections each time but in separate threads(pool of 40 and all threads will be used most of the time) and closing it when work in done. its a java process. Not always but sometimes close connection takes more than 10 seconds and observerd that particular message was n...
Vinay S Jain
1

votes
0

answer
470

Views

Embedded Kafka tests (ran by SBT) intermittently fails with ZooKeeperServer errors

I'm writing a group of tests using the spring KafkaEmbedded test util. The tests each individually stand up an embedded kafka instance, produce events and assert the resultant downstream events. The tests consistently pass when run in an IDE (e.g. IntelliJ), however the tests fail intermittently (ap...
Freestyle076

View additional questions