Thursday, July 15, 2010

Java ThreadPool and Large Training

A little shocking discovery we had on the Java ThreadPoolExecutor implementation: our large training cluster would somehow never filled up with a specific kind of training task. The culprit was eventually identified in the ThreadPoolExecutor, which gracefully queued up all tasks. I always thought new threads will be created when the queue is not empty and the number of threads is less than the maxPoolSize. Obviously, I did not read the fine-print in JavaDoc. There were quite a few different strategies described in ThreadPoolExecutor: Direct Handoffs (never queue, always create new thread till reach max), Unbounded queues (always queue, and never reach max threads) and Bounded queues (queue to a certain size, then start ceating threads till max threads, then rejects).

My assumption, Direct Handoffs+Unbounded Queue (use as many threads till reach maxPoolSize, then queue up unbounded), was obviously not one of them, even though I would think this is definitely favourable strategy in most cases. Anyway, we implemented this using a customized LinkedBlockingQueue.

Lessons learned: (like dealing with the bank,) always read the fine-print;).


  1. Do your guys use a pure java trainer for AMs or make system calls to binaries? That may make a quite big difference on the design of the parallelization platform. However I wonder if there exists toolkits for AM training in pure java...

  2. Works for me with such customized queue:

    static class ThreadPoolTaskQueue extends LinkedBlockingQueue{

    private ThreadPoolExecutor threadPool = null;

    public boolean offer(Runnable e) {
    if(threadPool != null && threadPool.getPoolSize() < threadPool.getMaximumPoolSize())
    return false;
    return super.offer(e);

    public void setThreadPool(ThreadPoolExecutor threadPool){
    this.threadPool = threadPool;

  3. Such a very useful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article.

  4. It is extremely beneficial to a newcomer like me. I'm a regular reader of your blog. You shared a very informative post here. Please keep blogging. if you require financial accounting homework help online contact this professional writing service. Stop wasting time and come to him right away for assistance with your assignment. You are aware that this company offers reasonable assignment help.

  5. Hey people! “Would you write my research paper for free from payforessay?” We will help you at a reasonable price. If you're having trouble getting started, this platform's assistant can help.