A little shocking discovery we had on the Java ThreadPoolExecutor implementation: our large training cluster would somehow never filled up with a specific kind of training task. The culprit was eventually identified in the ThreadPoolExecutor, which gracefully queued up all tasks. I always thought new threads will be created when the queue is not empty and the number of threads is less than the maxPoolSize. Obviously, I did not read the fine-print in JavaDoc. There were quite a few different strategies described in ThreadPoolExecutor: Direct Handoffs (never queue, always create new thread till reach max), Unbounded queues (always queue, and never reach max threads) and Bounded queues (queue to a certain size, then start ceating threads till max threads, then rejects).
My assumption, Direct Handoffs+Unbounded Queue (use as many threads till reach maxPoolSize, then queue up unbounded), was obviously not one of them, even though I would think this is definitely favourable strategy in most cases. Anyway, we implemented this using a customized LinkedBlockingQueue.
Lessons learned: (like dealing with the bank,) always read the fine-print;).
-
Hadoop! Hadoop! Hadoop!
ReplyDeleteI know, I know, I know...
ReplyDeleteDo your guys use a pure java trainer for AMs or make system calls to binaries? That may make a quite big difference on the design of the parallelization platform. However I wonder if there exists toolkits for AM training in pure java...
ReplyDeleteWorks for me with such customized queue:
ReplyDeletestatic class ThreadPoolTaskQueue extends LinkedBlockingQueue{
private ThreadPoolExecutor threadPool = null;
@Override
public boolean offer(Runnable e) {
if(threadPool != null && threadPool.getPoolSize() < threadPool.getMaximumPoolSize())
return false;
else
return super.offer(e);
}
public void setThreadPool(ThreadPoolExecutor threadPool){
this.threadPool = threadPool;
}
}
Hi, Great.. Tutorial is just awesome..It is really helpful for a newbie like me.. I am a regular follower of your blog. Really very informative post you shared here. Kindly keep blogging. If anyone wants to become a Java developer learn from Java Training in Chennai. or learn thru Java Online Training from India . Nowadays Java has tons of job opportunities on various vertical industry.
ReplyDeleteNice Information Your first-class knowledge of this great job can become a suitable foundation for these people. I did some research on the subject and found that almost everyone will agree with your blog.
ReplyDeleteCyber Security Course in Bangalore
Writing in style and getting good compliments on the article is hard enough, to be honest, but you did it so calmly and with such a great feeling and got the job done. This item is owned with style and I give it a nice compliment. Better!
ReplyDeleteCyber Security Training in Bangalore
Really, this article is truly one of the best, information shared was valuable and resourceful Very good work thank you.
ReplyDeleteData Scientist Training in Hyderabad
Vé máy bay đi Mỹ
ReplyDeletevé máy bay từ mỹ về việt nam
vé máy bay cần thơ phú quốc
vé máy bay đi nha trang tháng 7
Vé máy bay giá rẻ hà Nội đi Sài Gòn
Such a very useful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article.
ReplyDelete