EzDevInfo.com

Python multiprocessing - Pipe vs Queue

What are the fundamental differences between queues and pipes in Python's multiprocessing package?

In what scenarios should one choose one over the other? When is it advantageous to use Pipe()? When is it advantageous to use Queue()?

How do I instantiate a Queue object in java?

When I try:

Queue<Integer> q = new Queue<Integer>();

the compiler is giving me an error. Any help?

Also, if I want to initialize a queue do I have to implement the methods of the queue?

LinkedBlockingQueue vs ConcurrentLinkedQueue

My question relates to this question asked earlier. In situations where I am using a queue for communication between producer and consumer threads would people generally recommend using LinkedBlockingQueue or ConcurrentLinkedQueue?

What are the advantages / disadvantages of using one over the other?

The main difference I can see from an API perspective is that a LinkedBlockingQueue can be optionally bounded.

How do you implement a Stack and a Queue in JavaScript?

What is the best way to implement a Stack and a Queue in JavaScript?

I'm looking to do the shunting-yard algorithm and I'm going to need these data-structures.

How do I make and use a Queue in Objective-C?

I want to use a queue data structure in my Objective-C program. In C++ I'd use the STL queue. What is the equivalent data structure in Objective-C? How do I push/pop items?

Creating a blocking Queue in .NET?

I have a scenario where I have multiple threads adding to a queue and multiple threads reading from the same queue. If the queue reaches a specific size all threads that are filling the queue will be blocked on add until an item is removed from the queue.

The solution below is what I am using right now and my question is: How can this be improved? Is there an object that already enables this behavior in the BCL that I should be using?

internal class BlockingCollection<T> : CollectionBase, IEnumerable
{
    //todo: might be worth changing this into a proper QUEUE

    private AutoResetEvent _FullEvent = new AutoResetEvent(false);

    internal T this[int i]
    {
        get { return (T) List[i]; }
    }

    private int _MaxSize;
    internal int MaxSize
    {
        get { return _MaxSize; }
        set
        {
            _MaxSize = value;
            checkSize();
        }
    }

    internal BlockingCollection(int maxSize)
    {
        MaxSize = maxSize;
    }

    internal void Add(T item)
    {
        Trace.WriteLine(string.Format("BlockingCollection add waiting: {0}", Thread.CurrentThread.ManagedThreadId));

        _FullEvent.WaitOne();

        List.Add(item);

        Trace.WriteLine(string.Format("BlockingCollection item added: {0}", Thread.CurrentThread.ManagedThreadId));

        checkSize();
    }

    internal void Remove(T item)
    {
        lock (List)
        {
            List.Remove(item);
        }

        Trace.WriteLine(string.Format("BlockingCollection item removed: {0}", Thread.CurrentThread.ManagedThreadId));
    }

    protected override void OnRemoveComplete(int index, object value)
    {
        checkSize();
        base.OnRemoveComplete(index, value);
    }

    internal new IEnumerator GetEnumerator()
    {
        return List.GetEnumerator();
    }

    private void checkSize()
    {
        if (Count < MaxSize)
        {
            Trace.WriteLine(string.Format("BlockingCollection FullEvent set: {0}", Thread.CurrentThread.ManagedThreadId));
            _FullEvent.Set();
        }
        else
        {
            Trace.WriteLine(string.Format("BlockingCollection FullEvent reset: {0}", Thread.CurrentThread.ManagedThreadId));
            _FullEvent.Reset();
        }
    }
}

Is there a fixed sized queue which removes excessive elements?

I need a queue with a fixed size. When I add an element and the queue is full, it should automatically remove the oldest element.

Is there an existing implementation for this in Java?

Why is there no generic synchronized queue in .NET?

I noticed that you can call Queue.Synchronize to get a thread-safe queue object, but the same method isn't available on Queue<T>. Does anyone know why? Seems kind of weird.

Does the iOS SDK provide queues and stacks?

I'm writing an iPhone app, and I'm surprised that there seem to be no NSQueue or NSStack classes in Apple's Foundation Framework. I see that it would be quite easy to roll my own, starting with an NSMutableArray, so I'll do that unless I've missed something. Have I missed something?

Size-limited queue that holds last N elements in Java

A very simple & quick question on Java libraries: is there a ready-made class that implements a Queue with a fixed maximum size - i.e. it always allows addition of elements, but it will silently remove head elements to accomodate space for newly added elements.

Of course, it's trivial to implement it manually:

import java.util.LinkedList;

public class LimitedQueue<E> extends LinkedList<E> {
    private int limit;

    public LimitedQueue(int limit) {
        this.limit = limit;
    }

    @Override
    public boolean add(E o) {
        super.add(o);
        while (size() > limit) { super.remove(); }
        return true;
    }
}

As far as I see, there's no standard implementation in Java stdlibs, but may be there's one in Apache Commons or something like that?

Queue.Queue vs. collections.deque

I need a queue which multiple threads can put stuff into, and multiple threads may read from.

Python has at least two queue classes, Queue.Queue and collections.deque, with the former seemingly using the latter internally. Both claim to be thread-safe in the documentation.

However, the Queue docs also state:

collections.deque is an alternative implementation of unbounded queues with fast atomic append() and popleft() operations that do not require locking.

Which I guess I don't quite unterstand: Does this mean deque isn't fully thread-safe after all?

If it is, I may not fully understand the difference between the two classes. I can see that Queue adds blocking functionality. On the other hand, it loses some deque features like support for the in-operator.

Accessing the internal deque object directly, is

x in Queue().deque

thread-safe?

Also, why does Queue employ a mutex for it's operations when deque is thread-safe already?

Run PHP Task Asynchronously

I work on a somewhat large web application, and the backend is mostly in PHP. There are several places in the code where I need to complete some task, but I don't want to make the user wait for the result. For example, when creating a new account, I need to send them a welcome email. But when they hit the 'Finish Registration' button, I don't want to make them wait until the email is actually sent, I just want to start the process, and return a message to the user right away.

Up until now, in some places I've been using what feels like a hack with exec(). Basically doing things like:

exec("doTask.php $arg1 $arg2 $arg3 >/dev/null 2>&1 &");

Which appears to work, but I'm wondering if there's a better way. I'm considering writing a system which queues up tasks in a MySQL table, and a separate long-running PHP script that queries that table once a second, and executes any new tasks it finds. This would also have the advantage of letting me split the tasks among several worker machines in the future if I needed to.

Am I re-inventing the wheel? Is there a better solution than the exec() hack or the MySQL queue?

Java Queue implementations, which one?

From javadoc:

• A ConcurrentLinkedQueue is an appropriate choice when many threads will share access to a common collection. This queue does not permit null elements.
• ArrayBlockingQueue is a classic "bounded buffer", in which a fixed-sized array holds elements inserted by producers and extracted by consumers. This class supports an optional fairness policy for ordering waiting producer and consumer threads
• LinkedBlockingQueue typically have higher throughput than array-based queues but less predictable performance in most concurrent applications.

I have 2 scenarios, one requires the queue to support many producers (threads using it) with one consumer and the other is the other way arround.

I am not understanding whether I should use ConcurrentLikedQueue or the other ones (the array or linkedList implementations). Wherent' all this implementations supposed to be concurrent? I mean, can somebody explain me what is the diference between ConcurrentLikedQueue and LinkedBlockingQueue ?

Also, what is the optional fairness policy thing in the ArrayBlockingQueue please ?

Thanks a lot !

Implement a queue in which push_rear(), pop_front() and get_min() are all constant time operations

I came across this question: Implement a queue in which push_rear(), pop_front() and get_min() are all constant time operations.

I initially thought of using a min-heap data structure which has O(1) complexity for a get_min(). But push_rear() and pop_front() would be O(log(n)).

Does anyone know what would be the best way to implement such a queue which has O(1) push(), pop() and min()?

I googled about this, and wanted to point out this Algorithm Geeks thread. But it seems that none of the solutions follow constant time rule for all 3 methods: push(), pop() and min().

Thanks for all the suggestions.

In C# would it be better to use Queue.Synchronized or lock() for thread safety?

I have a Queue object that I need to ensure is thread-safe. Would it be better to use a lock object like this:

lock(myLockObject)
{
//do stuff with the queue
}

Or is it recommended to use Queue.Synchronized like this:

Queue.Synchronized(myQueue).whatever_i_want_to_do();

From reading the MSDN docs it says I should use Queue.Synchronized to make it thread-safe, but then it gives an example using a lock object. From the MSDN article:

To guarantee the thread safety of the Queue, all operations must be done through this wrapper only.

Enumerating through a collection is intrinsically not a thread-safe procedure. Even when a collection is synchronized, other threads can still modify the collection, which causes the enumerator to throw an exception. To guarantee thread safety during enumeration, you can either lock the collection during the entire enumeration or catch the exceptions resulting from changes made by other threads.

If calling Synchronized() doesn't ensure thread-safety what's the point of it? Am I missing something here?