Questions tagged [pyzmq]

1

votes
1

answer
254

Views

ZeroMQ multiple topics in publisher

List a and b are the keys and the values. The values in variable b are assigned as topicnumbers and data is sent every 10 seconds to a topic. Is this is a stable way to distribute information? Does one see any difficulties for in the near future when the keys and thus topics are scaled up to 100+? i...
Bondeaux
1

votes
1

answer
207

Views

Pyzmq - Send multiple messages without blocking

I want to send multiple messages from one socket to another and I encountered this problem. client.py context = Context() for i in range(10): print(i) out_socket = context.socket(REQ) out_socket.connect('tcp://localhost:%s' % '5000') message_content = ('hello', 1) pickled_message = dumps(message_con...
M.Puk
1

votes
1

answer
32

Views

How many bytes are used for longer string when sending via ZMQ?

I'm using ZeroMQ / ZMQ from Python and Java and have a question. When sending a shorter string, ZMQ uses one byte as described here (http://zguide.zeromq.org/page:all#A-Minor-Note-on-Strings) Then what goes onto the wire is a length (one byte for shorter strings) and the string contents as individua...
kangaroo
1

votes
0

answer
35

Views

Threading independent functions and waiting on them finishing

If I start a thread with start_new_thread, and have n independent functions that I want to run concurrently, I would do the following: def foo1(x): print 'foo1' time.sleep(5) def foo2(x): print 'foo2' time.sleep(1) func_list = [foo1,foo2] for k,j in enumerate(func_list): thread.start_new_thread(func...
ajsp
1

votes
0

answer
195

Views

json.loads and pickle.loads not usable with async keyword?

I have a zmq server that subscribes to several publisher (but in this example only to one) After making the server async by creating a coroutine and receiving a serialized multipart message deserialization with pickle.loads and json.loads both seem not to work. (I do not even get an error message)...
Rene Pickhardt
1

votes
0

answer
138

Views

ZeroMQ (via pyzmq) vs. Python Standard Library: Why one over the other?

Soon I'm going to be writing some Python code to make two services (A and B) talk to each other. They might be on the same machine, or they might not (but I will know in advance of deployment). I have two options; either the standard library, or ØMQ (via pyzmq). Why would I use one over the othe...
JesseTG
1

votes
0

answer
39

Views

Spawning a Process within a thread raises ZMQError: Interrupted system call

I have a multithreaded, multiprocess code. The action takes place inside a thread called in a loop every 5 seconds (reactor LoopingCall). the main action function spawns a multiprocessing.Process that has no interaction with zeromq library. Somehow spawning this process leads to a ZMQError: Interru...
apramc
1

votes
0

answer
26

Views

Why send message in diffrent time but receive mssage in the same time when use zmq delear/delear model

When I use DEALER-DEALER model to communicate, I find: server send messages in different time, but client receive messages in the same. For example (Python code): # server context = zmq.Context() sock_conn = self.context.socket(zmq.DEALER) sock_conn.bind('tcp://*:%d' % port) sock_conn.send('msg1') t...
836304831
1

votes
0

answer
25

Views

NetMQ DealerSocket fails to receive message

My Python3 server code: import zmq context = zmq.Context() router = context.socket(zmq.ROUTER) router.bind('tcp://*:1337') router.send_multipart([b'build_0', b'']) while True: pass My C# client code: /// /// Receive data from server. /// private void RecvFromServer() { // Init the socket. recvFrom...
seth.alter
1

votes
0

answer
18

Views

How do I stream 4 ports at the same time using zmq sockets while keeping track of where it came from?

Below is how I am currently streaming from multiple ports, the problem is I am receiving random packets, and they are not coming in one port than the next port when each port is continuously flowing. How would I reconfigure this so I can receive continuous packets from all ports? procname = 'STREAM'...
Bryce Simmons
1

votes
0

answer
35

Views

What is the purpose of `tempfile.NamedTemporaryFile` while creating a client using zmq?

I'm reading the source code of AMP and I'm confused about the purpose tempfile.NamedTemporaryFile while the code creates a client using zmq. Here is a sketch of how the code works: Server: import pexpect import sys import zmq from socket import gethostname python = sys.executable context = zmq.Conte...
meTchaikovsky
1

votes
0

answer
46

Views

Python ZMQ examples over WLAN network

For a project I need to communicate between C++ and Python via ZMQ over the WLAN network. If I use my C++ implementation, everything works fine. I just type in the IP+Port number at the client.bind('tcp:// ...) and I can send messages via WLAN. If I try the same with the Python Code, it does not wor...
muella91
1

votes
0

answer
33

Views

How to get response from server in MQ system (python)

I would like to make a chatting room server between chatting room user clients. The system server have to send all utterances to translator and wait the response result. I made this system with async web socket library. How can I deal with this waiting part from translator, if I change to async MQ s...
haha
1

votes
0

answer
30

Views

Memory keeps on growing publishing images via pyZMQ publisher socket

I am trying to code a publisher that takes images from a simulation through APIs and publish them through a tcp socket, so that multiple processes -not necessarily on the same machine- can access the images subscribing to the topic. Everything seems to work, since I can see the streamed images via t...
Marco Ciarambino
1

votes
1

answer
51

Views

ZMQ Context socket creation is MUCH slower on new computer

This was originally seen in Python, but has since been replicated in C++. Here is the unit test that distills down and replicates the behavior on my new laptop. These are just local socket connections. def test_zmq_publisher_duration(self): max_duration = 1.0 t0 = time.time() socket = zmq.Context.in...
e-holder
1

votes
0

answer
62

Views

Python Memory Leak with pyzmq and tonardo

my application use pyzmq work with tornado, main work is just like a task center. The memory will increase into 6G when task is busy, but when i stop task request and after all zmq task finished, the memory only reduce to 3.9g. i use gc.set_debug(DEBUG_LEAK) to check memory garbage and find many obj...
yihu li
1

votes
0

answer
39

Views

PYZMQ not returning

I have a programm using ZMQ REQ/REP socket to check if a specific module is running at a host. The wanted behavior is that if I get an answer from the module I do stuff and if not (i.e. if no ZMQ REQ socket is present at the specified port) then I want the programm to return. For the error case I e...
user_na
0

votes
1

answer
13

Views

Convert ByteArray to Image File received in Python from SoftwareBitmap frame on C# Windows.Graphic.Imaging

I have a requirement to get the VideoFrame.SoftwareBitMap from the Windows.Media.VideoFrame in C# and then convert this SOftwareBitmap to a ByteArray and then send the ByteArray using ZeroMQ to Python (using ZMQ) using TCP socket connection. I am receiving the bytearray from C# to Python but i am n...
11

votes
1

answer
203

Views

Why does (Py)ZeroMQ open so many Unix socket files?

I tried to monitor the number of open Unix socket files with lsof -U | wc -l while I executed this code: >>> import zmq # 1375 Unix socket files >>> c = zmq.Context() # 1377 Unix socket files >>> s = c.socket(zmq.PUSH) # 1383 Unix socket files >>> s.close() # 1381 Unix socket files >>> c.destroy() #...
Peque
1

votes
1

answer
500

Views

Why is this implementation of push/pull in zeromq not returning all messages

I have a client server pair in zeromq. What is strange is that the pull client only receives only every other message sent. Here is my implementation ## Push Server import zmq def post(): context = zmq.Context() socket = context.socket(zmq.PUSH) socket.bind('tcp://127.0.0.1:3333') socket.send('hello...
user1876508
1

votes
1

answer
785

Views

How get redundancy for forwarder in ZMQ PUB/SUB?

I tried and made run this example. It worked very well. I can add as many publishers or subscribers as I want, everything will communicate thanks to the forwarder. My concern is about redundancy. If the forwarder does not respond anymore, nothing's gonna work until forwarder is up. I found an examp...
zetoune
1

votes
2

answer
191

Views

zmq ventilator/worker/sink paradigm not working w/ subprocess

I am trying to replicate the ventilator/workers/sink paradigm described in the ZMQ guide. I have the same Python Ventilator, the same C++ worker as, and the same Python Sink as was described in the ZMQ examples. I want to launch the ventilator, workers, and sink from one main python script, so I c...
Kiran K.
1

votes
1

answer
845

Views

zeromq: why dealer sends one message and receives from one client?

I'm trying to understand my code's behavior. I'm using zeromq to create a server that sends a 'ping' and waits for 'pong' responses. What i'm seeing is that when I send a ping, only one client receives it. when I run this code and send 'ping' for the first time i receive: pong: A and when i run it a...
philip.roth
1

votes
1

answer
650

Views

Which ZeroMQ pattern is best of an asynchronous pair of sockets?

I have a server (running on Amazon) and a single client that connects to it. After a connect has been established the client and server exclusively communicate with each other and send messages. e.g. 1. Client -> Server 2. Client -> Server 3. Client Server 5. Client
nickponline
1

votes
1

answer
1.3k

Views

Tornado and ZeroMQ

I've been trying to follow the code in Eventloops and PyZMQ, specifically the following code: from tornado.ioloop import IOLoop from zmq.eventloop.ioloop import ZMQPoller loop = IOLoop(ZMQPoller()) I'm trying this and I get this exception: Traceback (most recent call last): File 'C:\Users\ACaron\Des...
André Caron
1

votes
1

answer
1.3k

Views

cross compile PyZMQ using some arm-gcc

I want to cross compile PyZmq for an ARM-Cortex-A8 processor (AM335). But in my case the compiler is being run with flags which are not available for this processor. I followed this and this post and taking into account that the mentioned patches are obsolete I'm currently following this procedure:...
frans
1

votes
1

answer
308

Views

running a 2nd zmq.eventloop.ioloop

I want to create a PyZMQ eventloop in a background thread, and have it work correctly with both standalone Python scripts and IPython scripts. (IPython uses PyZMQ eventloops located in the main thread, so this is causing me problems and why I want to start a private ioloop in a background thread.) I...
Jason S
1

votes
1

answer
1.1k

Views

Why does ZMQ router sometimes lose messages?

I have a machine hosting a router socket as such: router = zmq_ctx.socket(zmq.ROUTER) router.setsockopt(zmq.constants.ROUTER_HANDOVER, 1) router.bind(url) Then I have a number of machines making connections to it like this. dealer = zmq_ctx.socket(zmq.DEALER) dealer.setsockopt(zmq.IDENTITY, options....
nickponline
1

votes
1

answer
786

Views

ZMQ pair (for signaling) is blocking because of bad connection

I have two threads. One is a Worker Thread, the other a Communication Thread. The Worker Thread is reading data off a serial port, doing some processing, and then enqueueing the results to be sent to a server. The Communication Tthread is reading the results off the queue, and sending it. The challe...
Jono
1

votes
1

answer
277

Views

How to set high water mark to block on flooded recipient in pyzmq?

I'm confused by HWM behavior in PyZMQ. I would expect the following code to block Failing Example In [1]: import zmq In [2]: context = zmq.Context() In [3]: alice = context.socket(zmq.ROUTER) In [4]: alice.set_hwm(1) In [5]: alice.setsockopt(zmq.RCVHWM, 1) # just to be sure In [6]: alice.bind('ipc...
MRocklin
1

votes
1

answer
169

Views

What might cause a ZMQ router socket to block?

What are common causes for a ROUTER socket to block on a .send() to a DEALER socket in ZMQ? I use Python, but suspect that the answer would be the same regardless of language.
MRocklin
1

votes
1

answer
414

Views

How to catch SIGERM, SIGINT from a process running pyzmq?

I am using PYZMQ and it seems I cannot catch SIGTERM anymore. How do I do that for a graceful handling of SIGTERM ? Below is the code snippet: from time import sleep from signal import signal, SIGTERM, SIGINT from threading import Lock, Thread try: import cPickle as pickle except ImportError: import...
Nipun Talukdar
1

votes
1

answer
122

Views

Suitable Zmq messsage pattern for communicating processes

I'm using pyzmq library to establish a communication between two python processes on the same machine. Looking at the available messaging patterns offered by ZMQ it is not clear which one is suitable.In my case both processes need to send independently messages to one another at some random times. B...
steve
1

votes
1

answer
371

Views

Using ipython notebook via SSH gives GLIBC not found error

I am trying to run ipython notebook via SSH in a virtualenv previously created on the computer directly logged in (no SSH). When doing so the ipython notebook fails with: (venv2) [host]user: ipython notebook --nobrowser Traceback (most recent call last): File '.../venv2/bin/ipython', line 11, in sy...
pawni
1

votes
1

answer
419

Views

How to install pyzmq “--with-pgm”

I have zmq version 4.1.3 and pyzmq version 15.2.0 installed on my machine (I assume through pip but I dont remember now). I have a need to connect to a UDP epgm socket but get the error 'protocol not supported'. I have searched the vast expanses of the internet and have found the answer: 'build ze...
AlENeuman
1

votes
1

answer
196

Views

ZeroMQ: are PUB/SUB topic subscriptions cheap?

Problem: I have a number of file uploads coming via HTTP in parallel ( uploads receiver ). I'm storing them temporarily on a local disk. Another process ( uploads submitter ) gets notified about new uploads and does specific processing ( parsing, extracting metadata, uploading to S3 etc ). Once uplo...
Vovan Kuznetsov
1

votes
2

answer
578

Views

Use of pyzmq's logging handler in python

I want to introduce a zmq based logging into a Python program. As I was facing ZMQError: Address in use errors I decided to boil it down to a simple proof of concept. I was able to run the boiled down version, but not to receive any log entries. This is the code I used: Log Publisher: import time im...
user_na
1

votes
1

answer
58

Views

ZMQ crashes “randomly” in aiohttp web service

We have a aiohttp based web services which uses ZMQ to send jobs to workers and waits for the result. We are of course using the ZMQ eventloop, so we can wait for ZMQ sockets. 'Sometimes' the process crashes and we get this stack trace: ... await socket.send(z, flags=flags) File '/usr/local/lib/pyth...
Achim
1

votes
2

answer
182

Views

Task progress from a ZeroMQ worker

Fairly new to ZeroMQ. I have a simple REQ/REP queue like below. I am using PHP but that doesn't matter as any language binding would be fine for me. This is client to request a task $ctx = new ZMQContext(); $req = new ZMQSocket($ctx, ZMQ::SOCKET_REQ); $req->connect('tcp://localhost:5454'); $req->sen...
Waku-2
1

votes
1

answer
81

Views

pyzmq: Is there a shortcut for a “sendto”-like function for a ROUTER socket?

I have a ROUTER socket, which accepts requests, does some work and sends back a response: import zmq context = zmq.Context() socket = context.socket(zmq.ROUTER) socket.bind('tcp://*:1234') sender, _, content = socket.recv_multipart() reply = do_some_work(content) # Do something wi...
Avraham Shukron

View additional questions