ThreadPoolExecutor機制
一、概述
- ThreadPoolExecutor作為java.util.concurrent包對外提供基礎(chǔ)實現(xiàn)拾碌，以內(nèi)部線程池的形式對外提供管理任務(wù)執(zhí)行吐葱，線程調(diào)度，線程池管理等等服務(wù)校翔；
- Executors方法提供的線程服務(wù)弟跑，都是通過參數(shù)設(shè)置來實現(xiàn)不同的線程池機制。
- 先來了解其線程池管理的機制防症，有助于正確使用孟辑，避免錯誤使用導(dǎo)致嚴(yán)重故障哎甲。同時可以根據(jù)自己的需求實現(xiàn)自己的線程池
二、核心構(gòu)造方法講解
下面是ThreadPoolExecutor最核心的構(gòu)造方法
構(gòu)造方法參數(shù)講解
| 參數(shù)名 | 作用 |
|---|---|
| corePoolSize | 最大線程池大小 |
| maximumPoolSize | 最大線程池大小 |
| keepAliveTime | 線程池中超過corePoolSize數(shù)目的空閑線程最大存活時間扑浸；可以allowCoreThreadTimeOut(true)使得核心線程有效時間 |
| TimeUnit | keepAliveTime時間單位 |
| workQueue | 阻塞任務(wù)隊列 |
| threadFactory | 新建線程工廠 |
| RejectedExecutionHandler | 當(dāng)提交任務(wù)數(shù)超過maxmumPoolSize+workQueue之和時烧给，任務(wù)會交給RejectedExecutionHandler來處理 |
重點講解：
其中比較容易讓人誤解的是：corePoolSize，maximumPoolSize喝噪，workQueue之間關(guān)系础嫡。
- 當(dāng)線程池小于corePoolSize時，新提交任務(wù)將創(chuàng)建一個新線程執(zhí)行任務(wù)酝惧，即使此時線程池中存在空閑線程榴鼎。
- 當(dāng)線程池達(dá)到corePoolSize時，新提交任務(wù)將被放入workQueue中晚唇，等待線程池中任務(wù)調(diào)度執(zhí)行
- 當(dāng)workQueue已滿巫财，且maximumPoolSize>corePoolSize時，新提交任務(wù)會創(chuàng)建新線程執(zhí)行任務(wù)
- 當(dāng)提交任務(wù)數(shù)超過maximumPoolSize時哩陕，新提交任務(wù)由RejectedExecutionHandler處理
- 當(dāng)線程池中超過corePoolSize線程平项，空閑時間達(dá)到keepAliveTime時，關(guān)閉空閑線程
- 當(dāng)設(shè)置allowCoreThreadTimeOut(true)時悍及，線程池中corePoolSize線程空閑時間達(dá)到keepAliveTime也將關(guān)閉
線程管理機制圖示：
image
三闽瓢、Executors提供的線程池配置方案
- 構(gòu)造一個固定線程數(shù)目的線程池，配置的corePoolSize與maximumPoolSize大小相同心赶，同時使用了一個無界LinkedBlockingQueue存放阻塞任務(wù)扣讼，因此多余的任務(wù)將存在再阻塞隊列，不會由RejectedExecutionHandler處理
public static ExecutorService newFixedThreadPool(int nThreads) {
return new ThreadPoolExecutor(nThreads, nThreads,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>());
}
- 構(gòu)造一個緩沖功能的線程池缨叫，配置corePoolSize=0椭符，maximumPoolSize=Integer.MAX_VALUE，keepAliveTime=60s,以及一個無容量的阻塞隊列 SynchronousQueue耻姥，因此任務(wù)提交之后销钝，將會創(chuàng)建新的線程執(zhí)行；線程空閑超過60s將會銷毀
public static ExecutorService newCachedThreadPool() {
return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
60L, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>());
}
- 構(gòu)造一個只支持一個線程的線程池琐簇，配置corePoolSize=maximumPoolSize=1曙搬，無界阻塞隊列LinkedBlockingQueue；保證任務(wù)由一個線程串行執(zhí)行
public static ExecutorService newSingleThreadExecutor() {
return new FinalizableDelegatedExecutorService
(new ThreadPoolExecutor(1, 1,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>()));
}
- 構(gòu)造有定時功能的線程池鸽嫂，配置corePoolSize纵装，無界延遲阻塞隊列DelayedWorkQueue；有意思的是：maximumPoolSize=Integer.MAX_VALUE据某，由于DelayedWorkQueue是無界隊列橡娄，所以這個值是沒有意義的
public static ScheduledExecutorService newScheduledThreadPool(int corePoolSize) {
return new ScheduledThreadPoolExecutor(corePoolSize);
}
public static ScheduledExecutorService newScheduledThreadPool(
int corePoolSize, ThreadFactory threadFactory) {
return new ScheduledThreadPoolExecutor(corePoolSize, threadFactory);
}
public ScheduledThreadPoolExecutor(int corePoolSize,
ThreadFactory threadFactory) {
super(corePoolSize, Integer.MAX_VALUE, 0, TimeUnit.NANOSECONDS,
new DelayedWorkQueue(), threadFactory);
}
四、定制屬于自己的非阻塞線程池
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.RejectedExecutionHandler;
import java.util.concurrent.ThreadFactory;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
public class CustomThreadPoolExecutor {
private ThreadPoolExecutor pool = null;
/**
* 線程池初始化方法
*
* corePoolSize 核心線程池大小----10
* maximumPoolSize 最大線程池大小----30
* keepAliveTime 線程池中超過corePoolSize數(shù)目的空閑線程最大存活時間----30+單位TimeUnit
* TimeUnit keepAliveTime時間單位----TimeUnit.MINUTES
* workQueue 阻塞隊列----new ArrayBlockingQueue<Runnable>(10)====10容量的阻塞隊列
* threadFactory 新建線程工廠----new CustomThreadFactory()====定制的線程工廠
* rejectedExecutionHandler 當(dāng)提交任務(wù)數(shù)超過maxmumPoolSize+workQueue之和時,
* 即當(dāng)提交第41個任務(wù)時(前面線程都沒有執(zhí)行完,此測試方法中用sleep(100)),
* 任務(wù)會交給RejectedExecutionHandler來處理
*/
public void init() {
pool = new ThreadPoolExecutor(
10,
30,
30,
TimeUnit.MINUTES,
new ArrayBlockingQueue<Runnable>(10),
new CustomThreadFactory(),
new CustomRejectedExecutionHandler());
}
public void destory() {
if(pool != null) {
pool.shutdownNow();
}
}
public ExecutorService getCustomThreadPoolExecutor() {
return this.pool;
}
private class CustomThreadFactory implements ThreadFactory {
private AtomicInteger count = new AtomicInteger(0);
@Override
public Thread newThread(Runnable r) {
Thread t = new Thread(r);
String threadName = CustomThreadPoolExecutor.class.getSimpleName() + count.addAndGet(1);
System.out.println(threadName);
t.setName(threadName);
return t;
}
}
private class CustomRejectedExecutionHandler implements RejectedExecutionHandler {
@Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
// 記錄異常
// 報警處理等
System.out.println("error.............");
}
}
// 測試構(gòu)造的線程池
public static void main(String[] args) {
CustomThreadPoolExecutor exec = new CustomThreadPoolExecutor();
// 1.初始化
exec.init();
ExecutorService pool = exec.getCustomThreadPoolExecutor();
for(int i=1; i<100; i++) {
System.out.println("提交第" + i + "個任務(wù)!");
pool.execute(new Runnable() {
@Override
public void run() {
try {
Thread.sleep(3000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("running=====");
}
});
}
// 2.銷毀----此處不能銷毀,因為任務(wù)沒有提交執(zhí)行完,如果銷毀線程池,任務(wù)也就無法執(zhí)行了
// exec.destory();
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
方法中建立一個核心線程數(shù)為30個癣籽，緩沖隊列有10個的線程池挽唉。每個線程任務(wù)滤祖，執(zhí)行時會先睡眠3秒，保證提交10任務(wù)時瓶籽，線程數(shù)目被占用完匠童，再提交30任務(wù)時，阻塞隊列被占用完塑顺，汤求，這樣提交第41個任務(wù)是，會交給CustomRejectedExecutionHandler 異常處理類來處理严拒。
提交任務(wù)的代碼如下：
public void execute(Runnable command) {
if (command == null)
throw new NullPointerException();
/*
* Proceed in 3 steps:
*
* 1. If fewer than corePoolSize threads are running, try to
* start a new thread with the given command as its first
* task. The call to addWorker atomically checks runState and
* workerCount, and so prevents false alarms that would add
* threads when it shouldn't, by returning false.
*
* 2. If a task can be successfully queued, then we still need
* to double-check whether we should have added a thread
* (because existing ones died since last checking) or that
* the pool shut down since entry into this method. So we
* recheck state and if necessary roll back the enqueuing if
* stopped, or start a new thread if there are none.
*
* 3. If we cannot queue task, then we try to add a new
* thread. If it fails, we know we are shut down or saturated
* and so reject the task.
*/
int c = ctl.get();
if (workerCountOf(c) < corePoolSize) {
if (addWorker(command, true))
return;
c = ctl.get();
}
if (isRunning(c) && workQueue.offer(command)) {
int recheck = ctl.get();
if (! isRunning(recheck) && remove(command))
reject(command);
else if (workerCountOf(recheck) == 0)
addWorker(null, false);
}
else if (!addWorker(command, false))
reject(command);
}
注意：41以后提交的任務(wù)就不能正常處理了扬绪，因為，execute中提交到任務(wù)隊列是用的offer方法裤唠，如上面代碼挤牛，這個方法是非阻塞的，所以就會交給CustomRejectedExecutionHandler 來處理种蘸，所以對于大數(shù)據(jù)量的任務(wù)來說墓赴，這種線程池，如果不設(shè)置隊列長度會OOM航瞭，設(shè)置隊列長度诫硕，會有任務(wù)得不到處理，接下來我們構(gòu)建一個阻塞的自定義線程池
五沧奴、定制屬于自己的阻塞線程池
package com.tongbanjie.trade.test.commons;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.RejectedExecutionHandler;
import java.util.concurrent.ThreadFactory;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
public class CustomThreadPoolExecutor {
private ThreadPoolExecutor pool = null;
/**
* 線程池初始化方法
*
* corePoolSize 核心線程池大小----1
* maximumPoolSize 最大線程池大小----3
* keepAliveTime 線程池中超過corePoolSize數(shù)目的空閑線程最大存活時間----30+單位TimeUnit
* TimeUnit keepAliveTime時間單位----TimeUnit.MINUTES
* workQueue 阻塞隊列----new ArrayBlockingQueue<Runnable>(5)====5容量的阻塞隊列
* threadFactory 新建線程工廠----new CustomThreadFactory()====定制的線程工廠
* rejectedExecutionHandler 當(dāng)提交任務(wù)數(shù)超過maxmumPoolSize+workQueue之和時,
* 即當(dāng)提交第41個任務(wù)時(前面線程都沒有執(zhí)行完,此測試方法中用sleep(100)),
* 任務(wù)會交給RejectedExecutionHandler來處理
*/
public void init() {
pool = new ThreadPoolExecutor(
1,
3,
30,
TimeUnit.MINUTES,
new ArrayBlockingQueue<Runnable>(5),
new CustomThreadFactory(),
new CustomRejectedExecutionHandler());
}
public void destory() {
if(pool != null) {
pool.shutdownNow();
}
}
public ExecutorService getCustomThreadPoolExecutor() {
return this.pool;
}
private class CustomThreadFactory implements ThreadFactory {
private AtomicInteger count = new AtomicInteger(0);
@Override
public Thread newThread(Runnable r) {
Thread t = new Thread(r);
String threadName = CustomThreadPoolExecutor.class.getSimpleName() + count.addAndGet(1);
System.out.println(threadName);
t.setName(threadName);
return t;
}
}
private class CustomRejectedExecutionHandler implements RejectedExecutionHandler {
@Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
try {
// 核心改造點，由blockingqueue的offer改成put阻塞方法
executor.getQueue().put(r);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
// 測試構(gòu)造的線程池
public static void main(String[] args) {
CustomThreadPoolExecutor exec = new CustomThreadPoolExecutor();
// 1.初始化
exec.init();
ExecutorService pool = exec.getCustomThreadPoolExecutor();
for(int i=1; i<100; i++) {
System.out.println("提交第" + i + "個任務(wù)!");
pool.execute(new Runnable() {
@Override
public void run() {
try {
System.out.println(">>>task is running=====");
TimeUnit.SECONDS.sleep(10);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
});
}
// 2.銷毀----此處不能銷毀,因為任務(wù)沒有提交執(zhí)行完,如果銷毀線程池,任務(wù)也就無法執(zhí)行了
// exec.destory();
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
解釋：當(dāng)提交任務(wù)被拒絕時长窄，進(jìn)入拒絕機制滔吠，我們實現(xiàn)拒絕方法，把任務(wù)重新用阻塞提交方法put提交挠日，實現(xiàn)阻塞提交任務(wù)功能疮绷，防止隊列過大，OOM嚣潜，提交被拒絕方法在下面
public void execute(Runnable command) {
if (command == null)
throw new NullPointerException();
int c = ctl.get();
if (workerCountOf(c) < corePoolSize) {
if (addWorker(command, true))
return;
c = ctl.get();
}
if (isRunning(c) && workQueue.offer(command)) {
int recheck = ctl.get();
if (! isRunning(recheck) && remove(command))
reject(command);
else if (workerCountOf(recheck) == 0)
addWorker(null, false);
}
else if (!addWorker(command, false))
// 進(jìn)入拒絕機制冬骚， 我們把runnable任務(wù)拿出來，重新用阻塞操作put懂算，來實現(xiàn)提交阻塞功能
reject(command);
}
總結(jié)：
- 用ThreadPoolExecutor自定義線程池只冻，看線程是的用途，如果任務(wù)量不大计技，可以用無界隊列喜德，如果任務(wù)量非常大，要用有界隊列垮媒，防止OOM
- 如果任務(wù)量很大舍悯，還要求每個任務(wù)都處理成功航棱，要對提交的任務(wù)進(jìn)行阻塞提交，重寫拒絕機制萌衬，改為阻塞提交饮醇。保證不拋棄一個任務(wù)
- 最大線程數(shù)一般設(shè)為2N+1最好，N是CPU核數(shù)
- 核心線程數(shù)秕豫，看應(yīng)用朴艰，如果是任務(wù)，一天跑一次馁蒂，設(shè)置為0呵晚，合適，因為跑完就停掉了沫屡，如果是常用線程池饵隙，看任務(wù)量，是保留一個核心還是幾個核心線程數(shù)
- 如果要獲取任務(wù)執(zhí)行結(jié)果沮脖，用CompletionService金矛，但是注意，獲取任務(wù)的結(jié)果的要重新開一個線程獲取勺届，如果在主線程獲取驶俊，就要等任務(wù)都提交后才獲取，就會阻塞大量任務(wù)結(jié)果免姿，隊列過大OOM饼酿，所以最好異步開個線程獲取結(jié)果
