org.apache.curator.framework.recipes.queue

Class QueueSharder<U,T extends QueueBase<U>>

java.lang.Object

org.apache.curator.framework.recipes.queue.QueueSharder<U,T>

All Implemented Interfaces:

Closeable, AutoCloseable

public class QueueSharder<U,T extends QueueBase<U>>
extends Object
implements Closeable

A utility for shard a distributed queue.

Due to limitations in ZooKeeper's transport layer, a single queue will break if it has more than 10K-ish items in it. This class provides a facade over multiple distributed queues. It monitors the queues and if any one of them goes over a threshold, a new queue is added. Puts are distributed amongst the queues.

NOTE: item ordering is maintained within each managed queue but cannot be maintained across queues. i.e. items might get consumed out of order if they are in different managed queues.

Constructor Summary

Constructors

Constructor and Description
QueueSharder(org.apache.curator.framework.CuratorFramework client, QueueAllocator<U,T> queueAllocator, String queuePath, String leaderPath, QueueSharderPolicies policies)

Method Summary

Modifier and Type	Method and Description
void	close()
T	getQueue() Return one of the managed queues - the selection method cannot be relied on.
Collection<String>	getQueuePaths() Return the current set of shard paths
int	getShardQty() Return the current number of mananged queues
void	start() The sharder must be started

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

QueueSharder

public QueueSharder(org.apache.curator.framework.CuratorFramework client,
                    QueueAllocator<U,T> queueAllocator,
                    String queuePath,
                    String leaderPath,
                    QueueSharderPolicies policies)

Parameters:

client - client

queueAllocator - allocator for new queues

queuePath - path for the queues

leaderPath - path for the leader that monitors queue sizes (must be different than queuePath)

policies - sharding policies

Method Detail

start

public void start()
           throws Exception

The sharder must be started

Throws:

Exception - errors

close

public void close()

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

getQueue

public T getQueue()

Return one of the managed queues - the selection method cannot be relied on. It should be considered a random managed queue.

Returns:

a queue

getShardQty

public int getShardQty()

Return the current number of mananged queues

Returns:

qty

getQueuePaths

public Collection<String> getQueuePaths()

Return the current set of shard paths

Returns:

paths