一传黄、熔斷器(Circuit Breaker)介紹

熔斷器突想，現(xiàn)實生活中有一個很好的類比，就是家庭電路中都會安裝一個保險盒荚藻，當(dāng)電流過大的時候保險盒里面的保險絲會自動斷掉，來保護家里的各種電器及電路洁段。Hystrix中的熔斷器(Circuit Breaker)也是起到這樣的作用应狱，Hystrix在運行過程中會向每個commandKey對應(yīng)的熔斷器報告成功、失敗眉撵、超時和拒絕的狀態(tài)侦香，熔斷器維護計算統(tǒng)計的數(shù)據(jù)，根據(jù)這些統(tǒng)計的信息來確定熔斷器是否打開纽疟。如果打開罐韩，后續(xù)的請求都會被截斷。然后會隔一段時間默認(rèn)是5s污朽，嘗試半開散吵，放入一部分流量請求進(jìn)來，相當(dāng)于對依賴服務(wù)進(jìn)行一次健康檢查，如果恢復(fù)矾睦，熔斷器關(guān)閉晦款，隨后完全恢復(fù)調(diào)用。如下圖：

熔斷器開關(guān)圖.png

再來看下熔斷器在整個Hystrix流程圖中的位置，從步驟4開始赁温，如下圖：

Hystrix流程圖.png

Hystrix會檢查Circuit Breaker的狀態(tài)坛怪。如果Circuit Breaker的狀態(tài)為開啟狀態(tài)，Hystrix將不會執(zhí)行對應(yīng)指令股囊，而是直接進(jìn)入失敗處理狀態(tài)（圖中8 Fallback）袜匿。如果Circuit Breaker的狀態(tài)為關(guān)閉狀態(tài)，Hystrix會繼續(xù)進(jìn)行線程池稚疹、任務(wù)隊列居灯、信號量的檢查（圖中5）

二、如何使用熔斷器(Circuit Breaker)

由于Hystrix是一個容錯框架内狗，因此我們在使用的時候怪嫌，要達(dá)到熔斷的目的只需配置一些參數(shù)就可以了。但我們要達(dá)到真正的效果其屏，就必須要了解這些參數(shù)喇勋。Circuit Breaker一共包括如下6個參數(shù)。
1偎行、circuitBreaker.enabled
是否啟用熔斷器川背，默認(rèn)是TURE。
2蛤袒、circuitBreaker.forceOpen
熔斷器強制打開熄云，始終保持打開狀態(tài)。默認(rèn)值FLASE妙真。
3缴允、circuitBreaker.forceClosed
熔斷器強制關(guān)閉，始終保持關(guān)閉狀態(tài)珍德。默認(rèn)值FLASE练般。
4、circuitBreaker.errorThresholdPercentage
設(shè)定錯誤百分比锈候，默認(rèn)值50%薄料，例如一段時間（10s）內(nèi)有100個請求，其中有55個超時或者異常返回了泵琳，那么這段時間內(nèi)的錯誤百分比是55%摄职，大于了默認(rèn)值50%誊役，這種情況下觸發(fā)熔斷器-打開。
5谷市、circuitBreaker.requestVolumeThreshold
默認(rèn)值20.意思是至少有20個請求才進(jìn)行errorThresholdPercentage錯誤百分比計算蛔垢。比如一段時間（10s）內(nèi)有19個請求全部失敗了。錯誤百分比是100%迫悠，但熔斷器不會打開鹏漆，因為requestVolumeThreshold的值是20. 這個參數(shù)非常重要，熔斷器是否打開首先要滿足這個條件及皂，源代碼如下

// check if we are past the statisticalWindowVolumeThreshold
if (health.getTotalRequests() < properties.circuitBreakerRequestVolumeThreshold().get()) {
    // we are not past the minimum volume threshold for the statisticalWindow so we'll return false immediately and not calculate anything
    return false;
}

if (health.getErrorPercentage() < properties.circuitBreakerErrorThresholdPercentage().get()) {
    return false;
}

6甫男、circuitBreaker.sleepWindowInMilliseconds
半開試探休眠時間，默認(rèn)值5000ms验烧。當(dāng)熔斷器開啟一段時間之后比如5000ms，會嘗試放過去一部分流量進(jìn)行試探又跛，確定依賴服務(wù)是否恢復(fù)碍拆。

測試代碼（模擬10次調(diào)用，錯誤百分比為5%的情況下慨蓝，打開熔斷器開關(guān)感混。）：

package myHystrix.threadpool;

import com.netflix.hystrix.*;
import org.junit.Test;

import java.util.Random;

/**
 * Created by wangxindong on 2017/8/15.
 */
public class GetOrderCircuitBreakerCommand extends HystrixCommand<String> {

    public GetOrderCircuitBreakerCommand(String name){
        super(Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey("ThreadPoolTestGroup"))
                .andCommandKey(HystrixCommandKey.Factory.asKey("testCommandKey"))
                .andThreadPoolKey(HystrixThreadPoolKey.Factory.asKey(name))
                .andCommandPropertiesDefaults(
                        HystrixCommandProperties.Setter()
                                .withCircuitBreakerEnabled(true)//默認(rèn)是true，本例中為了展現(xiàn)該參數(shù)
                                .withCircuitBreakerForceOpen(false)//默認(rèn)是false礼烈，本例中為了展現(xiàn)該參數(shù)
                                .withCircuitBreakerForceClosed(false)//默認(rèn)是false弧满，本例中為了展現(xiàn)該參數(shù)
                                .withCircuitBreakerErrorThresholdPercentage(5)//(1)錯誤百分比超過5%
                                .withCircuitBreakerRequestVolumeThreshold(10)//(2)10s以內(nèi)調(diào)用次數(shù)10次，同時滿足(1)(2)熔斷器打開
                                .withCircuitBreakerSleepWindowInMilliseconds(5000)//隔5s之后此熬，熔斷器會嘗試半開(關(guān)閉)庭呜，重新放進(jìn)來請求
//                                .withExecutionTimeoutInMilliseconds(1000)
                )
                .andThreadPoolPropertiesDefaults(
                        HystrixThreadPoolProperties.Setter()
                                .withMaxQueueSize(10)   //配置隊列大小
                                .withCoreSize(2)    // 配置線程池里的線程數(shù)
                )
        );
    }

    @Override
    protected String run() throws Exception {
        Random rand = new Random();
        //模擬錯誤百分比(方式比較粗魯?shù)梢宰C明問題)
        if(1==rand.nextInt(2)){
//            System.out.println("make exception");
            throw new Exception("make exception");
        }
        return "running:  ";
    }

    @Override
    protected String getFallback() {
//        System.out.println("FAILBACK");
        return "fallback: ";
    }

    public static class UnitTest{

        @Test
        public void testCircuitBreaker() throws Exception{
            for(int i=0;i<25;i++){
                Thread.sleep(500);
                HystrixCommand<String> command = new GetOrderCircuitBreakerCommand("testCircuitBreaker");
                String result = command.execute();
                //本例子中從第11次，熔斷器開始打開
                System.out.println("call times:"+(i+1)+"   result:"+result +" isCircuitBreakerOpen: "+command.isCircuitBreakerOpen());
                //本例子中5s以后犀忱，熔斷器嘗試關(guān)閉募谎，放開新的請求進(jìn)來
            }
        }
    }
}

測試結(jié)果：

call times:1 result:fallback: isCircuitBreakerOpen: false
call times:2 result:running: isCircuitBreakerOpen: false
call times:3 result:running: isCircuitBreakerOpen: false
call times:4 result:fallback: isCircuitBreakerOpen: false
call times:5 result:running: isCircuitBreakerOpen: false
call times:6 result:fallback: isCircuitBreakerOpen: false
call times:7 result:fallback: isCircuitBreakerOpen: false
call times:8 result:fallback: isCircuitBreakerOpen: false
call times:9 result:fallback: isCircuitBreakerOpen: false
call times:10 result:fallback: isCircuitBreakerOpen: false
熔斷器打開
call times:11 result:fallback: isCircuitBreakerOpen: true
call times:12 result:fallback: isCircuitBreakerOpen: true
call times:13 result:fallback: isCircuitBreakerOpen: true
call times:14 result:fallback: isCircuitBreakerOpen: true
call times:15 result:fallback: isCircuitBreakerOpen: true
call times:16 result:fallback: isCircuitBreakerOpen: true
call times:17 result:fallback: isCircuitBreakerOpen: true
call times:18 result:fallback: isCircuitBreakerOpen: true
call times:19 result:fallback: isCircuitBreakerOpen: true
call times:20 result:fallback: isCircuitBreakerOpen: true
5s后熔斷器關(guān)閉
call times:21 result:running: isCircuitBreakerOpen: false
call times:22 result:running: isCircuitBreakerOpen: false
call times:23 result:fallback: isCircuitBreakerOpen: false
call times:24 result:running: isCircuitBreakerOpen: false
call times:25 result:running: isCircuitBreakerOpen: false

三、熔斷器(Circuit Breaker)源代碼HystrixCircuitBreaker.java分析

HystrixCircuitBreaker.java.png

Factory 是一個工廠類阴汇，提供HystrixCircuitBreaker實例

public static class Factory {
        //用一個ConcurrentHashMap來保存HystrixCircuitBreaker對象
        private static ConcurrentHashMap<String, HystrixCircuitBreaker> circuitBreakersByCommand = new ConcurrentHashMap<String, HystrixCircuitBreaker>();
        
//Hystrix首先會檢查ConcurrentHashMap中有沒有對應(yīng)的緩存的斷路器数冬，如果有的話直接返回。如果沒有的話就會新創(chuàng)建一個HystrixCircuitBreaker實例搀庶，將其添加到緩存中并且返回
        public static HystrixCircuitBreaker getInstance(HystrixCommandKey key, HystrixCommandGroupKey group, HystrixCommandProperties properties, HystrixCommandMetrics metrics) {
            
            HystrixCircuitBreaker previouslyCached = circuitBreakersByCommand.get(key.name());
            if (previouslyCached != null) {
                return previouslyCached;
            }

            
            HystrixCircuitBreaker cbForCommand = circuitBreakersByCommand.putIfAbsent(key.name(), new HystrixCircuitBreakerImpl(key, group, properties, metrics));
            if (cbForCommand == null) {
                return circuitBreakersByCommand.get(key.name());
            } else {
                return cbForCommand;
            }
        }

        
        public static HystrixCircuitBreaker getInstance(HystrixCommandKey key) {
            return circuitBreakersByCommand.get(key.name());
        }

        static void reset() {
            circuitBreakersByCommand.clear();
        }
}

HystrixCircuitBreakerImpl是HystrixCircuitBreaker的實現(xiàn)拐纱，allowRequest()、isOpen()哥倔、markSuccess()都會在HystrixCircuitBreakerImpl有默認(rèn)的實現(xiàn)秸架。

static class HystrixCircuitBreakerImpl implements HystrixCircuitBreaker {
        private final HystrixCommandProperties properties;
        private final HystrixCommandMetrics metrics;

        /* 變量circuitOpen來代表斷路器的狀態(tài)，默認(rèn)是關(guān)閉 */
        private AtomicBoolean circuitOpen = new AtomicBoolean(false);

        /* 變量circuitOpenedOrLastTestedTime記錄著斷路恢復(fù)計時器的初始時間未斑，用于Open狀態(tài)向Close狀態(tài)的轉(zhuǎn)換 */
        private AtomicLong circuitOpenedOrLastTestedTime = new AtomicLong();

        protected HystrixCircuitBreakerImpl(HystrixCommandKey key, HystrixCommandGroupKey commandGroup, HystrixCommandProperties properties, HystrixCommandMetrics metrics) {
            this.properties = properties;
            this.metrics = metrics;
        }

        /*用于關(guān)閉熔斷器并重置統(tǒng)計數(shù)據(jù)*/
        public void markSuccess() {
            if (circuitOpen.get()) {
                if (circuitOpen.compareAndSet(true, false)) {
                    //win the thread race to reset metrics
                    //Unsubscribe from the current stream to reset the health counts stream.  This only affects the health counts view,
                    //and all other metric consumers are unaffected by the reset
                    metrics.resetStream();
                }
            }
        }

        @Override
        public boolean allowRequest() {
            //是否設(shè)置強制開啟
            if (properties.circuitBreakerForceOpen().get()) {
                return false;
            }
            if (properties.circuitBreakerForceClosed().get()) {//是否設(shè)置強制關(guān)閉
                isOpen();
                // properties have asked us to ignore errors so we will ignore the results of isOpen and just allow all traffic through
                return true;
            }
            return !isOpen() || allowSingleTest();
        }

        public boolean allowSingleTest() {
            long timeCircuitOpenedOrWasLastTested = circuitOpenedOrLastTestedTime.get();
            //獲取熔斷恢復(fù)計時器記錄的初始時間circuitOpenedOrLastTestedTime咕宿，然后判斷以下兩個條件是否同時滿足：
            // 1) 熔斷器的狀態(tài)為開啟狀態(tài)(circuitOpen.get() == true)
            // 2) 當(dāng)前時間與計時器初始時間之差大于計時器閾值circuitBreakerSleepWindowInMilliseconds(默認(rèn)為 5 秒)
            //如果同時滿足的話币绩，表示可以從Open狀態(tài)向Close狀態(tài)轉(zhuǎn)換。Hystrix會通過CAS操作將circuitOpenedOrLastTestedTime設(shè)為當(dāng)前時間府阀，并返回true缆镣。如果不同時滿足，返回false试浙，代表熔斷器關(guān)閉或者計時器時間未到董瞻。
            if (circuitOpen.get() && System.currentTimeMillis() > timeCircuitOpenedOrWasLastTested + properties.circuitBreakerSleepWindowInMilliseconds().get()) {
                // We push the 'circuitOpenedTime' ahead by 'sleepWindow' since we have allowed one request to try.
                // If it succeeds the circuit will be closed, otherwise another singleTest will be allowed at the end of the 'sleepWindow'.
                if (circuitOpenedOrLastTestedTime.compareAndSet(timeCircuitOpenedOrWasLastTested, System.currentTimeMillis())) {
                    // if this returns true that means we set the time so we'll return true to allow the singleTest
                    // if it returned false it means another thread raced us and allowed the singleTest before we did
                    return true;
                }
            }
            return false;
        }

        @Override
        public boolean isOpen() {
            if (circuitOpen.get()) {//獲取斷路器的狀態(tài)
                // if we're open we immediately return true and don't bother attempting to 'close' ourself as that is left to allowSingleTest and a subsequent successful test to close
                return true;
            }

            // Metrics數(shù)據(jù)中獲取HealthCounts對象
            HealthCounts health = metrics.getHealthCounts();

            // 檢查對應(yīng)的請求總數(shù)(totalCount)是否小于屬性中的請求容量閾值circuitBreakerRequestVolumeThreshold,默認(rèn)20，如果是的話表示熔斷器可以保持關(guān)閉狀態(tài)田巴，返回false
            if (health.getTotalRequests() < properties.circuitBreakerRequestVolumeThreshold().get()) {
                
                return false;
            }

            //不滿足請求總數(shù)條件钠糊，就再檢查錯誤比率(errorPercentage)是否小于屬性中的錯誤百分比閾值(circuitBreakerErrorThresholdPercentage，默認(rèn) 50),如果是的話表示斷路器可以保持關(guān)閉狀態(tài)壹哺，返回 false
            if (health.getErrorPercentage() < properties.circuitBreakerErrorThresholdPercentage().get()) {
                return false;
            } else {
                // 如果超過閾值抄伍，Hystrix會判定服務(wù)的某些地方出現(xiàn)了問題，因此通過CAS操作將斷路器設(shè)為開啟狀態(tài)管宵，并記錄此時的系統(tǒng)時間作為定時器初始時間截珍，最后返回 true
                if (circuitOpen.compareAndSet(false, true)) {
                    circuitOpenedOrLastTestedTime.set(System.currentTimeMillis());
                    return true;
                } else {
                    return true;
                }
            }
        }

    }

四、總結(jié)

每個熔斷器默認(rèn)維護10個bucket,每秒一個bucket,每個blucket記錄成功,失敗,超時,拒絕的狀態(tài)箩朴，默認(rèn)錯誤超過50%且10秒內(nèi)超過20個請求進(jìn)行中斷攔截岗喉。下圖顯示HystrixCommand或HystrixObservableCommand如何與HystrixCircuitBreaker及其邏輯和決策流程進(jìn)行交互，包括計數(shù)器在斷路器中的行為炸庞。

熔斷器流程交互圖.png

轉(zhuǎn)載請注明出處钱床，并附上鏈接http://www.reibang.com/p/14958039fd15
參考資料：https://github.com/Netflix/Hystrix/wiki