kue
Kue is a priority job queue backed by redis, built for node.js.
Kue
I can't figure out what I'm doing wrong, perhaps somone can point it out. I'm trying to figure out why my 'job complete' event isn't firing.
var kue = require('kue'),
jobs = kue.createQueue();
var util= require('util');
var job = jobs.create('test', util.puts('123')).on('complete', function(){
console.log("Job complete");
}).on('failed', function(){
console.log("Job failed");
}).on('progress', function(progress){
process.stdout.write('\r job #' + job.id + ' ' + progress + '% complete');
});
Now when I run this on node it prints 123 but it doesn't say job complete.
Source: (StackOverflow)
I am using kue for delayed jobs in my node.js application.
I have some problems to figure out how I can restart a job using the API of kue without having to move the id of a job manually from the the list of failed jobs to the list of inactive jobs using redis commands.
Is this possible using kue?
I don't want to set a fixed number of retry attempts - I just want to retry specific jobs.
Suggestions for a well maintained alternative to kue are also welcome.
Source: (StackOverflow)
Using Kue, how do I schedule a job to be executed once every Thursday? The Kue readme mentions that I can delay a Job, but what about repeatedly executing the Job at a specific time?
I can do what I want with a cron job, but I like Kue's features.
What I want is to process a Job once anytime on Thursday, but only once.
Source: (StackOverflow)
I setup a kue job as I usually do :
var job = jobs.create('phase2', s);
job.on('complete', function (){
console.log('Job'+job.id+' is done');
}).on('failed', function (){
console.log('Job'+job.id+'has failed');
});
job.save();
On certain conditions, I want the job to fail and restart automatically. For that, I have in my processor the following lines :
if(t==1){
//Keep going
}else{
console.log('PROBLEM');
job.failed();
}
I have tried changing the failed event to :
.on('failed', function (){
console.log('Job'+job.id+'has failed');
job.state('inactive').save();
});
as suggested here : Node.js Kue how to restart failed jobs
I have also tried adding attempts() as in the Readme, like so :
var job = jobs.create('phase2', s).attempts(5);
None of these things have worked. The job is marked as 'failed' and my console shows 'PROBLEM', but I do not see the message defined in the 'failed' listener.
EDIT
After more reading, i've tried listening to the 'job failed' event at queue level, using :
jobs.on('job failed', function(id,result){
console.log('fail queue');
kue.Job.get(id, function(err, job){
job.state('inactive').save();
});
});
Same result, and no console log... It seem to be something else than Kue, any idea what could make the events not fire?
Source: (StackOverflow)
I would like to test an application that uses Kue so that job queue is empty before each test and cleared after each test. Queue should be fully functional and I need to be able to check status of jobs that are already in the queue.
I tried mock-kue and it worked well until I had to get jobs from the queue and analyze them. I couldn't get it to return jobs by job ID.
Situations that I need to be able to test:
- Something happens and there should be a job of a given type in the queue,
- Something happens and produces a job. Something else happens and that job gets removed and replaced with another job (rescheduling or existing job).
Seams straightforward, but I have hard time wrapping my head around the problem. All pointers are welcome.
Source: (StackOverflow)
I'm using mongoose and Kue for the Flow control. I pass an object retrieved from the database to Kue. When the job get processed, the object hasn't anymore some functions, like .save() & others.
jobs.process('process', 5, function(job, done) {
var url = job.data.url;
var objo = job.data.comp;
request({uri:url, json:true}, function(err, res, body) {
objo.meta = body;
// Here it throw an error that save is note defined
// TypeError: Object #<Object> has no method 'save'
objo.save(function(err) {
if (err)
throw err;
console.log('Saved data for ' + objo.title);
done();
});
});
});
var q = db.Entity.find({}).sort('_id', 1).limit(10);
q.execFind(function(err, docs) {
docs.forEach(function(objo) {
jobs.create('process', {
comp : objo,
url : 'http://example.com/' + encodeURIComponent(objo.permalink) + '.js'
}).save();
})
});
Thanks in advance
Source: (StackOverflow)
I am getting an redis connection error when depolyed in APP Fog. I don't know if KUE is generating too many connections.
Here is stack
/mnt/var/vcap.local/dea/apps/kue-0-543205581cf47e8551d042ab06df92e1/app/node_mod
ules/redis/index.js:516
throw callback_err;
^
Error: Auth error: ERR max number of clients reached
at Command.RedisClient.do_auth.self.send_anyway as callback
at RedisClient.return_error (/mnt/var/vcap.local/dea/apps/kue-0-543205581cf4
7e8551d042ab06df92e1/app/node_modules/redis/index.js:512:25)
at ReplyParser.RedisClient.init_parser (/mnt/var/vcap.local/dea/apps/kue-0-5
43205581cf47e8551d042ab06df92e1/app/node_modules/redis/index.js:263:14)
at ReplyParser.EventEmitter.emit (events.js:96:17)
at ReplyParser.send_error (/mnt/var/vcap.local/dea/apps/kue-0-543205581cf47e
8551d042ab06df92e1/app/node_modules/redis/lib/parser/javascript.js:296:10)
at ReplyParser.execute (/mnt/var/vcap.local/dea/apps/kue-0-543205581cf47e855
1d042ab06df92e1/app/node_modules/redis/lib/parser/javascript.js:181:22)
at RedisClient.on_data (/mnt/var/vcap.local/dea/apps/kue-0-543205581cf47e855
1d042ab06df92e1/app/node_modules/redis/index.js:488:27)
at Socket. (/mnt/var/vcap.local/dea/apps/kue-0-543205581cf47e8551
d042ab06df92e1/app/node_modules/redis/index.js:82:14)
at Socket.EventEmitter.emit (events.js:96:17)
at TCP.onread (net.js:396:14)
The redis server log shows it made 5 client connections. is there a way to reduce the no of connections
Thanks
Source: (StackOverflow)
I am using kue in one of my project to remove stale locks from the database. I am able to process individual jobs, like updating individual documents in mongoDB.
var kue = require('kue')
, queue = kue.createQueue();
queue.process('staleLocks', function(job, done){
removeLock(job.data.id, done);
});
function removeLock(id, done) {
// mongoDB call
done();
}
But, I want to update all the documents at once by collecting all the ids aftet the queue has been processed. I am able to even store all the ids. The only problem is that am not able to track, when the queue has been processed, some sort of event to synchronize the whole process.
Source: (StackOverflow)
My code is leaking memory. After a couple of hours, it fills up the entire memory and crashes. I've simplified my code here, would anybody be able to tell if this looks like it leaks? Thanks.
var request = require('request').forever(), // as per [1]
async = require('async'),
kue = require('kue'),
jobs = kue.createQueue(),
pool = { maxSockets: 1 };
function main (job, done) {
async.series(
[function (callback) {
var req = request({url: job.data.URL1, pool: pool}, function (err, resp, body) {
//stuff...
callback(err);
});
},
function (callback) {
var req = request({url: job.data.URL2}, function (err, resp, body) {
//stuff...
callback(err);
});
}
],
function (err) {
//stuff...
done();
}
);
}
jobs.process('job_name', function (job, done) { //many jobs with 'job_name' in the queue
main (job, done);
});
[1] https://groups.google.com/d/msg/nodejs/ZI6WnDgwwV0/sFm4QKK7ODEJ
Source: (StackOverflow)
Basically, the user is logged in on the meteor app running on localhost:30000 using the accounts-ui package. Express app is running on localhost:34444.
At some point, a user will need to use the express app (it's actually Kue), and I want to avoid having users login twice.
Both instances are running on same domain, just different ports.
My end goal is to have a job queue management) accessible by a user logged on from another web server instance (in this case meteor), all in the same domain.
Source: (StackOverflow)
I am building a web app with node.js and mongodb.
I need to add delayed jobs. For example, sending users an email one month after signing up.
I haven't found an existing solution for delayed jobs besides Kue, but it uses Redis, and I would prefer to use my existing Mongodb before adding another resource to my web app.
Is there any existing solution for that?
Source: (StackOverflow)
I have a very simple logical question.
I will be running job processing logic on a separate app server.
My job processing app will be a standalone app, doing nothing just processing jobs.
In my code, how do I make sure that my app continuously keep checking redis server for jobs ?
-Do I need to run the code in infinite loop ?
-or Do I need to keep restarting my app
or there is some inbuilt mechanism in Kue that I'm missing here ?
Thanks
Source: (StackOverflow)
My app makes use of Kue to queue up requests, which are handled by worker.js because I need to send the requests that the job makes through Proximo - it's a little confusing. But because of this, the results of the job are unable to be sent back to the user. Previously the job would res.send(results) and then the user would have the data.
What would be the best way to get the Kue'ed job to send results back to the user? The only way I can think of at the moment is to use a web hook, but it's not the most efficient way of doing so and it builds a wall between the user and their data.
Source: (StackOverflow)
I'm using kue which uses node_redis, but I'm also already using node_redis for my sessions, so I'd like to have kue create a server on a specific port say the default 6379 and then kue listen on port 1234.
How would I be able to do this? I found this article which talks about something similar, but I don't really want to have to create an init script to do this.
Source: (StackOverflow)
From the kue docs,creating a queue and adding job is is easy but i cannot follow how the job is being stored
var kue = require('kue')
jobs = kue.createQueue();
adding a job
jobs.create('email', {
title: 'welcome email for tj'
, to: 'tj@learnboost.com'
, template: 'welcome-email'
}).priority('high').save();
The example is easy to understand but,what if i needed more options for instance in the example,like adding an advert option - , ad: 'we are the best'
jobs.create('email', {
title: 'welcome email for tj'
, to: 'tj@learnboost.com'
, ad: 'we are the best'
, template: 'welcome-email'
}).priority('high').save();
how am i going to go about it?.
Source: (StackOverflow)