A little shocking discovery we had on the Java ThreadPoolExecutor implementation: our large training cluster would somehow never filled up with a specific kind of training task. The culprit was eventually identified in the ThreadPoolExecutor, which gracefully queued up all tasks. I always thought new threads will be created when the queue is not empty and the number of threads is less than the maxPoolSize. Obviously, I did not read the fine-print in JavaDoc. There were quite a few different strategies described in ThreadPoolExecutor: Direct Handoffs (never queue, always create new thread till reach max), Unbounded queues (always queue, and never reach max threads) and Bounded queues (queue to a certain size, then start ceating threads till max threads, then rejects).
My assumption, Direct Handoffs+Unbounded Queue (use as many threads till reach maxPoolSize, then queue up unbounded), was obviously not one of them, even though I would think this is definitely favourable strategy in most cases. Anyway, we implemented this using a customized LinkedBlockingQueue.
Lessons learned: (like dealing with the bank,) always read the fine-print;).
-
Hadoop! Hadoop! Hadoop!
ReplyDeleteI know, I know, I know...
ReplyDeleteDo your guys use a pure java trainer for AMs or make system calls to binaries? That may make a quite big difference on the design of the parallelization platform. However I wonder if there exists toolkits for AM training in pure java...
ReplyDeleteWorks for me with such customized queue:
ReplyDeletestatic class ThreadPoolTaskQueue extends LinkedBlockingQueue{
private ThreadPoolExecutor threadPool = null;
@Override
public boolean offer(Runnable e) {
if(threadPool != null && threadPool.getPoolSize() < threadPool.getMaximumPoolSize())
return false;
else
return super.offer(e);
}
public void setThreadPool(ThreadPoolExecutor threadPool){
this.threadPool = threadPool;
}
}
Vé máy bay đi Mỹ
ReplyDeletevé máy bay từ mỹ về việt nam
vé máy bay cần thơ phú quốc
vé máy bay đi nha trang tháng 7
Vé máy bay giá rẻ hà Nội đi Sài Gòn
Such a very useful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article.
ReplyDeleteIt is extremely beneficial to a newcomer like me. I'm a regular reader of your blog. You shared a very informative post here. Please keep blogging. if you require financial accounting homework help online contact this professional writing service. Stop wasting time and come to him right away for assistance with your assignment. You are aware that this company offers reasonable assignment help.
ReplyDeleteHey people! “Would you write my research paper for free from payforessay?” We will help you at a reasonable price. If you're having trouble getting started, this platform's assistant can help.
ReplyDeleteAmazing Post Vist here for fun
ReplyDeleteyoutube shorts download
Java Thread Pools are quite a useful utility for handling concurrent tasks particularly when dealing with large training procedures. Thus, saving up on thread utilization, they cut down on overhead and increase speed on your academic work, could benefit from cheap dissertation writing to manage your deadlines effectively
ReplyDeleteThis training should really help developers finding new way of doing things! Our developer at alloy wheels dubai is also practicing the code
ReplyDelete
ReplyDeleteExplore a wide variety of dildos at Cupidbaba, designed for ultimate pleasure and satisfaction. Choose from different shapes, sizes, and materials to suit your preferences. Perfect for solo play or partnered experiences, offering exciting sensations every time.