jzmq
Java binding for ZeroMQ
Distributed Messaging - zeromq
Trying to follow the directions from: http://github.com/zeromq/jzmq
I installed pkg-config using Homebrew and then I run the following commands:
./autogen.sh
./configure
The configure fails with:
checking how to hardcode library paths into programs... immediate
./configure: line 15263: syntax error near unexpected token `newline'
./configure: line 15263: ` PKG_CHECK_MODULES('
Source: (StackOverflow)
Hello all I'm attempting to follow the directions located at: https://github.com/nathanmarz/storm/wiki/Installing-native-dependencies for installing Zero MQ as a dependency for Storm on a Ubuntu 12.04 machine. However when trying to run the make command I get the following error
Making all in src
make[1]: Entering directory `/home/localadmin/jzmq/src'
make[1]: *** No rule to make target `classdist_noinst.stamp', needed by `org/zeromq/ZMQ.class'. Stop.
make[1]: Leaving directory `/home/localadmin/jzmq/src'
make: *** [all-recursive] Error 1
Does anyone have any idea where this error stems from and how I would be able to correct it?
Source: (StackOverflow)
I'm trying to program with ZeroMQ on Ubuntu using the Java examples. I've successfully built ZeroMq core and Java bindings, but trying to run simple example, it just hangs indefinitely.
Here's what I've done so far, btw, I'm using JDK/JRE 1.7, ZeroMQ 3.2.2, and Ubuntu 12.10 64-bit
Build ZeroMQ Core Libraries
I downloaded core ZeroMQ 3.2.2 POSIX tarball from here, extracted and built according to instructions here. Configure, make, and installation all went well, no errors, I also ran ldconfig following make to update the system library cache. A quick look at /usr/local/lib and the core libraries appear to have built correctly...
usr1@ubuntu:/usr/local/lib$ ls -ltr *.so
lrwxrwxrwx 1 root dev 15 Apr 1 19:31 libzmq.so -> libzmq.so.3.0.0
lrwxrwxrwx 1 root dev 16 Apr 1 20:14 libjzmq.so -> libjzmq.so.0.0.0
Build JZMQ Java Bindings
Downloaded Java bindings source from here, configured, ran make; all looks good. This creates zmq.jar and zmq-perf.jar; ran ldconfig again to update library cache; moved jars to /usr/local/share/java...
usr1@ubuntu:/usr/local/share/java$ ls -ltr
-rw-r--r-- 1 usr1 usr1 40507 Apr 1 20:14 zmq.jar
-rw-rw-r-- 1 usr1 usr1 4809 Apr 2 08:41 zmq-perf.jar
Running Example local_lat
I ran the local_lat example detailed here using the following settings for my environment...
java -Djava.library.path=/usr/local/lib -classpath /usr/local/share/java/zmq-perf.jar:/usr/local/share/java/zmq.jar local_lat tcp://127.0.0.1:5555 1 100
Got UnsatisfiedLinkError; fixed that by just logging in and out.
Ran again and it just hangs, no errors, nata, zilch. I removed the argument 100 from above and got the usage message from local_lat, so I know it's finding the zmq jar files since local_lat class resides in zmq-perf.jar. Ran again with the above command, and it just hangs; no errors, nothing. A bit of a let down since it was quite a bit of work to get everything built.
UPDATE
See below, turns out this was a problem with how I was testing
Source: (StackOverflow)
The question is in the title but to elaborate a bit. If I'm writing an NIO application in Java using the Sun/Oracle NIO APIs or a framework like Netty, is it possible to have a client "connect" as a subscriber even while there is no server bound to the host/port it connects to? What I effectively want to do is just not care if the server is dead but as soon as it is online and sends a message I receive it as if it was there the whole time. Take this ZMQ server and client for e.g.
Starting the client first....
import org.zeromq.ZMQ;
import java.util.Date;
public class ZMQClient {
public static void main(String[] args) {
// Prepare our context and subscriber
ZMQ.Context context = ZMQ.context(1);
ZMQ.Socket subscriber = context.socket(ZMQ.SUB);
subscriber.connect("tcp://localhost:5563");
subscriber.subscribe("".getBytes());
while (true) {
// Read envelope with address
String address = new String(subscriber.recv(0));
// Read message contents
String contents = new String(subscriber.recv(0));
System.out.println(address + " : " + contents+" - "+ new Date());
}
}
}
...and some time later the server
import org.zeromq.ZMQ;
import java.util.Date;
public class ZMQServer {
public static void main(String[] args) throws Exception{
// Prepare our context and publisher
ZMQ.Context context = ZMQ.context(1);
ZMQ.Socket publisher = context.socket(ZMQ.PUB);
publisher.bind("tcp://127.0.0.1:5563");
while (true) {
// Write two messages, each with an envelope and content
publisher.send("".getBytes(), ZMQ.SNDMORE);
publisher.send("We don't want to see this".getBytes(), 0);
publisher.send("".getBytes(), ZMQ.SNDMORE);
publisher.send("We would like to see this".getBytes(), 0);
System.out.println("Sent @ "+new Date());
Thread.sleep(1000);
}
}
}
Source: (StackOverflow)
I am facing a strange issue with ZMQ, which I'm just not able to debug. These are the components:
- Java ZMQ Server - Almost an exact copy of this example. There are a hundred worker threads.
PHP Client - Simple request reply with a REQ socket. This is the request flow:
$zcontext = new ZMQContext();
$socket = new ZMQSocket($zcontext, ZMQ::SOCKET_REQ);
$socket->connect(<address>);
$startTime = microtime(true);
$socket->send(<request>);
$result = $socket->recv();
$totalTime = microtime(true) - $startTime;
The ZMQ sockets use TCP and both the server and client are on the same machine.
The PHP script is served by apache and I am load testing using apache benchmark. I make 5000 requests with a concurrency of 200. On the PHP client I log the time it takes for the request reply ($totalTime). In most of the cases, this time is very low (sub 500ms), but occasionally it takes a really long time - sometimes even 60 secs (for send + receive).
I added some extra logging to find out where the issue is happening, and it turns out that whenever it takes really long, almost all the time is between PHP's send and Java's receive - so packets are taking really long to reach the server.
I'm not setting any special ZMQ settings, or otherwise doing anything unusual so I don't know what is causing the issue. It should also be noted that the issue persists even at lower concurrencies (I tested at 100 and 150 too), but the max request times are lower.
Sorry if the question seems vague - I'll provide any other details that may be needed.
Source: (StackOverflow)
I have a java application using ZMQ. I've been able to run it on my Win7 PC where I placed the jzmq.dll in the same folder where the jar executable is, then I run it by the command "java -jar myapp.jar".
My next step is to move it to run on a server. So I copied it all over to the server (Win 2008), the files as well as the directory structure. Apparently the very same command that gets it to work on my PC doesn't work on the server.
Any ideas why and what should I do to get it run on the server as well?
Following the first comment below I've copied to the server all the ZMQ related folders I have, preserving their directory structure. Still getting the same error. More details on the error message are:
Source: (StackOverflow)
I wrote a simple test in java (JDK 7) for a ZeroMQ PUB socket publishing data over a MULTICAST channel on Windows 7 using OpenPGM 5.2.122. I tried JZMQ versions 2.2.0, 2.1.3 and 2.1.0 on top of ZeroMQ 3.2.3. The test file is as below.
import org.zeromq.ZMQ;
public class ZMQMulticastPubSocketTest
{
public static void main(String[] args)
{
ZMQ.Context ctx = ZMQ.context(1);
ZMQ.Socket pub = ctx.socket(ZMQ.PUB);
pub.setLinger(0);
pub.setRate(10000000);
pub.setSendBufferSize(24000000);
pub.connect("epgm://10.100.20.19;239.9.9.11:5556");
//pub.bind("tcp://*:5556");
while(true)
{
pub.sendMore("TESTTOPIC");
pub.send("Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_".getBytes(), 0);
}
}
}
I notice that the memory footprint of the process keeps increasing till the computer runs out of memory. It doesn't crash (I am sure malloc() fails are internally handled). I also tried it on our linux servers and it went all the way to consuming 22 GB of ram before I took the process down. Is there a memory leak in the JZMQ wwrapper for multicast?
If I changed the code above to bind to a TCP address (line commented out) the memory footprint stayed stable and barely increased.
I also wrote a C version of the above code. This version is given below and it did not have the same growing memory footprint issue for multicast.
#include "stdafx.h"
#include "zmq.h"
#include "zmq_utils.h"
#include <assert.h>
#include <string>
static int
s_send (void *socket, char *string) {
int size = zmq_send (socket, string, strlen (string), 0);
return size;
}
static int
s_sendmore (void *socket, char *string) {
int size = zmq_send (socket, string, strlen (string), ZMQ_SNDMORE);
return size;
}
int main(int argc, char* argv[])
{
void *context = zmq_ctx_new ();
void *publisher = zmq_socket (context, ZMQ_PUB);
int rc = zmq_bind (publisher, "epgm://10.100.20.19;239.9.9.11:5556");
assert (rc == 0);
long sockOpt = 1000000;
rc = zmq_setsockopt (publisher, ZMQ_RATE, &sockOpt, sizeof(sockOpt));
sockOpt = 0;
rc = zmq_setsockopt (publisher, ZMQ_LINGER, &sockOpt, sizeof(sockOpt));
sockOpt = 24000000;
rc = zmq_setsockopt (publisher, ZMQ_SNDBUF, &sockOpt, sizeof(sockOpt));
char* topic = "TESTTOPIC";
while(1)
{
s_sendmore(publisher, topic);
s_send(publisher, "Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_Data_");
}
return 0;
}
Does anyone have any idea about why this may be happening?
Thanks.
Source: (StackOverflow)
I have zmq.jar built by someone else long time ago. How can I find out which version it corresponds to?
Not much in the MANIFEST:
$ cat MANIFEST.MF
Manifest-Version: 1.0
Created-By: 1.6.0_14 (Sun Microsystems Inc.)
Here's the contents. Would someone be able to tell whether it's 2.x or 3.x?
$ jar tvf zmq.jar
0 Thu Feb 02 14:59:52 EST 2012 META-INF/
71 Thu Feb 02 14:59:52 EST 2012 META-INF/MANIFEST.MF
2429 Wed Feb 01 14:24:32 EST 2012 org/zeromq/App.class
4320 Tue Jan 24 14:40:32 EST 2012 org/zeromq/EmbeddedLibraryTools.class
2392 Tue Jan 24 14:40:32 EST 2012 org/zeromq/ZContext.class
3536 Tue Jan 24 14:40:32 EST 2012 org/zeromq/ZFrame.class
920 Tue Jan 24 14:40:32 EST 2012 org/zeromq/ZMQ$Context.class
2401 Tue Jan 24 14:40:32 EST 2012 org/zeromq/ZMQ$Error.class
3232 Tue Jan 24 14:40:32 EST 2012 org/zeromq/ZMQ$Poller.class
5613 Tue Jan 24 14:40:32 EST 2012 org/zeromq/ZMQ$Socket.class
2484 Tue Jan 24 14:40:32 EST 2012 org/zeromq/ZMQ.class
771 Tue Jan 24 14:40:32 EST 2012 org/zeromq/ZMQException.class
1468 Tue Jan 24 14:40:32 EST 2012 org/zeromq/ZMQForwarder.class
1663 Tue Jan 24 14:40:32 EST 2012 org/zeromq/ZMQQueue.class
424 Tue Jan 24 14:40:32 EST 2012 org/zeromq/ZMQStreamer.class
9771 Tue Jan 24 14:40:32 EST 2012 org/zeromq/ZMsg.class
Source: (StackOverflow)
I have written a request-reply broker using zeromq and the C programming language. The broker routes client requests to the appropriate services, and then routes the reply back to the client. The services are written in JAVA.
Can someone please explain how to have the services communicate with the broker. I am sure that this must be a common scenario, but I don't have much experience, so can someone please help me with making my code inter-operable.
Please assume that the services will not be zeromq aware. Is node.js to be used in such a scenario? Will I have to write an http front end?
Source: (StackOverflow)
I'm experimenting with java flavored zmq to test the benefits of using PGM over TCP in my project. So I changed the weather example, from the zmq guide, to use the epgm transport.
Everything compiles and runs, but nothing is being sent or received. If I change the transport back to TCP, the server receives the messages sent from the client and I get the console output I'm expecting.
So, what are the requirements for using PGM? I changed the string, that I'm passing to the bind and connect methods, to follow the zmq api for zmq_pgm: "transport://interface;multicast address:port". That didn't work. I get and invalid argument error whenever I attempt to use this format. So, I simplified it by dropping the interface and semicolon which "works", but I'm not getting any results.
I haven't been able to find a jzmq example that uses pgm/epgm and the api documentation for the java binding does not define the appropriate string format for an endpoint passed to bind or connect. So what am I missing here? Do I have to use different hosts for the client and the server?
One thing of note is that I'm running my code on a VirtualBox VM (Ubuntu 14.04/OSX Mavericks host). I'm not sure if that has anything to do with the issue I'm currently facing.
Server:
public class wuserver {
public static void main (String[] args) throws Exception {
// Prepare our context and publisher
ZMQ.Context context = ZMQ.context(1);
ZMQ.Socket publisher = context.socket(ZMQ.PUB);
publisher.bind("epgm://xx.x.x.xx:5556");
publisher.bind("ipc://weather");
// Initialize random number generator
Random srandom = new Random(System.currentTimeMillis());
while (!Thread.currentThread ().isInterrupted ()) {
// Get values that will fool the boss
int zipcode, temperature, relhumidity;
zipcode = 10000 + srandom.nextInt(10000) ;
temperature = srandom.nextInt(215) - 80 + 1;
relhumidity = srandom.nextInt(50) + 10 + 1;
// Send message to all subscribers
String update = String.format("%05d %d %d", zipcode, temperature, relhumidity);
publisher.send(update, 0);
}
publisher.close ();
context.term ();
}
}
Client:
public class wuclient {
public static void main (String[] args) {
ZMQ.Context context = ZMQ.context(1);
// Socket to talk to server
System.out.println("Collecting updates from weather server");
ZMQ.Socket subscriber = context.socket(ZMQ.SUB);
//subscriber.connect("tcp://localhost:5556");
subscriber.connect("epgm://xx.x.x.xx:5556");
// Subscribe to zipcode, default is NYC, 10001
String filter = (args.length > 0) ? args[0] : "10001 ";
subscriber.subscribe(filter.getBytes());
// Process 100 updates
int update_nbr;
long total_temp = 0;
for (update_nbr = 0; update_nbr < 100; update_nbr++) {
// Use trim to remove the tailing '0' character
String string = subscriber.recvStr(0).trim();
StringTokenizer sscanf = new StringTokenizer(string, " ");
int zipcode = Integer.valueOf(sscanf.nextToken());
int temperature = Integer.valueOf(sscanf.nextToken());
int relhumidity = Integer.valueOf(sscanf.nextToken());
total_temp += temperature;
}
System.out.println("Average temperature for zipcode '"
+ filter + "' was " + (int) (total_temp / update_nbr));
subscriber.close();
context.term();
}
}
Source: (StackOverflow)
I'm trying to use Java client with ZeroMQ. When subscribing to any prefix, the Java client matches no messages, although a similar Python client matches messages as expected.
The Python server
context = zmq.Context()
socket = context.socket(zmq.PUB)
socket.bind("tcp://*:5556")
for i in range(100):
r = "XXX " + i
socket.send_string(r)
time.sleep(random.randint(0,10))
The Python client working fine
context = zmq.Context()
socket = context.socket(zmq.SUB)
socket.connect("tcp://localhost:5556")
zip_filter = "XXX"
socket.setsockopt_string(zmq.SUBSCRIBE, zip_filter)
for update_nbr in range(5):
s = socket.recv_string()
print(s)
The Java client matching no messages
context = ZMQ.context(1);
subscriber = context.socket(ZMQ.SUB);
subscriber.connect("tcp://localhost:5556");
String filter = "XXX";
subscriber.subscribe(filter.getBytes(Charset.forName("UTF-8")));
while (true) {
String msg = subscriber.recvStr(0, Charset.forName("UTF-8"));
// ...
}
Using the above Python server, the Python client matches all messages starting with XXX as expected.
Using the same Python server, the Java client matches no messages.
Do you have any idea what is wrong with the call to subscribe() in the Java client?
Source: (StackOverflow)
I successfully created publisher but failed to create subscriber by using the following:
public static void main(String [] args)
{
ActorSystem system = ActorSystem.create("System");
ActorRef subscriber = system.actorOf(new Props(Sub.class), "subscriber");
subscriber.tell(new MyActor("CharlieParker", 50, 25), subscriber);
}
public class Sub extends UntypedActor
{
ActorRef subSocket = ZeroMQExtension.get(getContext().system()).newSubSocket(
new Connect("tcp://127.0.0.1:1237"),
new Listener(getSelf()), Subscribe.all());
}
Got this error:
Uncaught error from thread [System-akka.zeromq.socket-dispatcher-7] shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled for ActorSystem[System]
java.lang.NoSuchMethodError: org.zeromq.ZMQ$Poller.poll(J)J
at akka.zeromq.ConcurrentSocketActor$$anonfun$10.apply(ConcurrentSocketActor.scala:180)
at akka.zeromq.ConcurrentSocketActor$$anonfun$10.apply(ConcurrentSocketActor.scala:179)
at akka.zeromq.ConcurrentSocketActor.akka$zeromq$ConcurrentSocketActor$$doPoll(ConcurrentSocketActor.scala:197)
at akka.zeromq.ConcurrentSocketActor$$anonfun$receive$1.applyOrElse(ConcurrentSocketActor.scala:46)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:425)
at akka.actor.ActorCell.invoke(ActorCell.scala:386)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:230)
at akka.dispatch.Mailbox.run(Mailbox.scala:212)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
What does it mean?
Source: (StackOverflow)
I installed zmq using homebrew and then installed jzmq from the github master.
I am getting this error:
java.lang.UnsatisfiedLinkError: no jzmq in java.library.path
I tried setting the path:
System.setProperty("java.library.path","/usr/local/lib")
These files are in /usr/local/lib
libzmq.a
libzmq.dylib
libzmq.la
libjzmq.a
libjzmq.dylib
libjzmq.la
Source: (StackOverflow)
I am trying to use ZeroMQ's pub-sub sockets. However, I don't clearly understand the role of context (zmq::context_t) while creating sockets (zmq::socket_t).
Assuming that I want to create 5 subscriber sockets (zmq::socket_t using ZMQ_SUB), do I need 5 contexts, one for each of the subscriber sockets? Or can I use a single context for all 5 sockets?
Source: (StackOverflow)