Hystrix入門指南

Introduction

 

一、Where does the name come from?

hystrix對應的中文名字是「豪豬」，豪豬周身長滿了刺，能保護本身不受天敵的傷害，表明了一種防護機制，這與hystrix自己的功能不謀而合，所以Netflix團隊將該框架命名爲Hystrix，並使用了對應的卡通形象作做爲logo。

二、What Is Hystrix?

在一個分佈式系統裏，許多依賴不可避免的會調用失敗，好比超時、異常等，如何可以保證在一個依賴出問題的狀況下，不會致使總體服務失敗，這個就是Hystrix須要作的事情。Hystrix提供了熔斷、隔離、Fallback、cache、監控等功能，可以在一個、或多個依賴同時出現問題時保證系統依然可用。

 

   

三、Hello Hystrix

 1 public class CommandHelloWorld extends HystrixCommand<String> {
 2  
 3     private final String name;
 4  
 5     public CommandHelloWorld(String name) {
 6         super(HystrixCommandGroupKey.Factory.asKey("ExampleGroup")); //必須
 7         this.name = name;
 8     }
 9  
10     @Override
11     protected String run() {
12         /*
13          網絡調用 或者其餘一些業務邏輯，可能會超時或者拋異常
14         */
15         return "Hello " + name + "!";
16     }
17 }
18   
19 String s = new CommandHelloWorld("Bob").execute(); //
20 Future<String> s = new CommandHelloWorld("Bob").queue();
21 Observable<String> s = new CommandHelloWorld("Bob").observe();
22 Observable<String> s = new CommandHelloWorld("Bob").toObservable()

    說明：

• execute() — blocks, then returns the single response received from the dependency (or throws an exception in case of an error)
• queue() — returns a Future with which you can obtain the single response from the dependency
• observe() — subscribes to the Observable that represents the response(s) from the dependency and returns an Observable that replicates that source Observable
• toObservable() — returns an Observable that, when you subscribe to it, will execute the Hystrix command and emit its responses

 

四、Flow Chart

 

 說明：

1. Construct a HystrixCommand or HystrixObservableCommand Object
2. Execute the Command
3. Is the Response Cached?
4. Is the Circuit Open?
5. Is the Thread Pool/Queue/Semaphore Full?
6. HystrixObservableCommand.construct() or HystrixCommand.run()
7. Calculate Circuit Health
8. Get the Fallback
9. Return the Successful Response

 經常使用功能介紹

依賴隔離

一個用戶請求的成功執行，肯能依賴數十上百個外部服務，若是沒有隔離，單個依賴的失敗，可能會印象其餘依賴的正常執行。以下圖所示，爲每一個依賴配置了單獨線程池

   

在下圖中，當Dep I 出現問題時，DepA 和Dep M大以來能夠正常執行

 線程池隔離的使用例子

 1 public class CommandHelloWorld extends HystrixCommand<String> {
 2  
 3     private final String name;
 4  
 5     public CommandHelloWorld(String name) {
 6         super(Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey("ExampleGroup"))  //必須
 7                 .andThreadPoolKey(HystrixThreadPoolKey.Factory.asKey("ExampleGroup-pool"))  //可選,默認 使用 this.getClass().getSimpleName();
 8                 .andThreadPoolPropertiesDefaults(HystrixThreadPoolProperties.Setter().withCoreSize(4)));
 9          
10         this.name = name;
11     }
12  
13     @Override
14     protected String run() throws InterruptedException {
15         System.out.println("running");
16         TimeUnit.MILLISECONDS.sleep(1000);
17         return "Hello " + name + "!";
18     }
19      
20 }

線程池經常使用參數設置：

實現類：HystrixThreadPoolProperties

名稱	類型	含義	默認值
coreSize	Integer	線程池大小	10
maxQueueSize	Integer	隊列大小，一經初始化後不能修改	-1
queueSizeRejectionThreshold	Integer	隊列reject閾值，能夠動態修改 maxQueueSize>0是生效，通常設置爲小於 maxQueueSizede 的數值	5

 

 

 

 

 

 

 

 

 

Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey("ExampleGroup"))  //必須
 .andCommandPropertiesDefaults(HystrixCommandProperties.Setter()
                .withExecutionTimeoutInMilliseconds(500))  //超時時間
 .andThreadPoolKey(HystrixThreadPoolKey.Factory.asKey("ExampleGroup-pool"))  //可選,默認 使用 this.getClass().getSimpleName();
 .andThreadPoolPropertiesDefaults(HystrixThreadPoolProperties.Setter().withCoreSize(4)
        .withMaxQueueSize(10).withQueueSizeRejectionThreshold(7))

 

Q： 怎麼設置線程池大小？

  A：Qps* Tp99 +冗餘線程

信號量隔離

線程池隔離中，發起請求的線程和真實執行的線程不是同一個線程，使用信號量隔離時，它們是同一個線程， 兩種隔離的區別以下圖：

 1 public class CommandUsingSemaphoreIsolation extends HystrixCommand<String> {
 2  
 3     private final int id;
 4     private long start,end ;
 5  
 6     public CommandUsingSemaphoreIsolation(int id) {
 7         super(Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey("ExampleGroup"))
 8                 // since we're doing an in-memory cache lookup we choose SEMAPHORE isolation
 9                 .andCommandPropertiesDefaults(HystrixCommandProperties.Setter()
10                         .withExecutionIsolationStrategy(HystrixCommandProperties.ExecutionIsolationStrategy.SEMAPHORE) //設置使用信號量隔離策略
11                                 .withExecutionIsolationSemaphoreMaxConcurrentRequests(3)  //設置信號量隔離時的最大併發請求數
12                                 .withFallbackIsolationSemaphoreMaxConcurrentRequests(5)     //設置fallback的最大併發數
13                         .withExecutionTimeoutInMilliseconds(300)));   //設置超時時間
14         this.id = id;
15         this.start = System.currentTimeMillis();
16     }
17  
18     @Override
19     protected String run() throws InterruptedException {
20         // a real implementation would retrieve data from in memory data structure
21         TimeUnit.MILLISECONDS.sleep(id*30);
22         System.out.println("running normal, id="+id);
23         return "ValueFromHashMap_" + id;
24     }
25  
26     @Override
27     protected String getFallback(){
28         System.out.println(" fallback, id="+id);
29         return "fallback:"+id;
30     }
31  
32 }
33   
34 @Test
35 public void maxCurrentRequst() throws InterruptedException {
36     int count =10;
37     while (count >0){
38         int id = count--;
39         new Thread(() -> {
40             try {
41                 new CommandUsingSemaphoreIsolation(id).execute();
42             }catch (Exception ex){
43                 System.out.println("Exception:"+ex.getMessage()+" id="+id);
44             }
45         }).start();
46     }
47  
48     TimeUnit.SECONDS.sleep(100);
49 }
50  //注：使用信號量隔離，在同一個線程中即便循環調用new CommandUsingSemaphoreIsolation(id).queue()，run方法也是順序執行;

//控制檯輸出

fallback, id=10
fallback, id=9
fallback, id=5
fallback, id=8
fallback, id=1
Exception:CommandUsingSemaphoreIsolation fallback execution rejected. id=4
Exception:CommandUsingSemaphoreIsolation fallback execution rejected. id=7
running normal, id=2
running normal, id=3
running normal, id=6

Q: 何時使用線程池隔離，什麼使用信號量隔離？

A:  線程池隔離缺點是帶來必定的開銷，但不會阻塞請求線程，適合於於IO密集型的任務

信號量隔離使用用戶請求線程，沒有格外線程切換開銷，使用與執行時間和執行邏輯都比較短的本地計算。好比CPU密集型的任務

Fallback

Q1: 爲何須要fallback?

簡單來講，在依賴調用失敗時，咱們通常會須要提供降級方案，Hystrix對此提供了支持

降級  

 1 public class CommandHelloWorld extends HystrixCommand<String> {
 2  
 3     private final String name;
 4  
 5     public CommandHelloWorld(String name) {
 6         super(Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey("ExampleGroup"))  //必須
 7                 .andCommandPropertiesDefaults(HystrixCommandProperties.Setter()
 8                         .withExecutionTimeoutInMilliseconds(500))  //超時時間
 9                 .andThreadPoolKey(HystrixThreadPoolKey.Factory.asKey("ExampleGroup-pool"))  //可選,默認 使用 this.getClass().getSimpleName();
10                 .andThreadPoolPropertiesDefaults(HystrixThreadPoolProperties.Setter().withCoreSize(4)));
11  
12         this.name = name;
13     }
14  
15     @Override
16     protected String run() throws InterruptedException {
17         System.out.println("running");
18         TimeUnit.MILLISECONDS.sleep(1000);
19         return "Hello " + name + "!";
20     }
21  
22     @Override
23     protected String getFallback() {
24         return "Hello "+"Fallback";
25     }
26 }
27   
28 @Test
29 public void fallbackTest(){
30     assertEquals("Hello Fallback",new CommandHelloWorld("World").execute());
31 }

 Q2：什麼狀況下會觸發fallback？

簡單來講，就是run方法拋異常，超時，線程/信號量reject、短路

Failure Type	Exception class	Exception.cause	subject to fallback
FAILURE	HystrixRuntimeException	underlying exception (user-controlled)	YES
TIMEOUT	HystrixRuntimeException	j.u.c.TimeoutException	YES
SHORT_CIRCUITED	HystrixRuntimeException	j.l.RuntimeException	YES
THREAD_POOL_REJECTED	HystrixRuntimeException	j.u.c.RejectedExecutionException	YES
SEMAPHORE_REJECTED	HystrixRuntimeException	j.l.RuntimeException	YES
BAD_REQUEST	HystrixBadRequestException	underlying exception (user-controlled)	NO

 

 

 

 

 

 

如下爲測試的主程序：

 1 public class CommandHelloFailure extends HystrixCommand<String> {
 2  
 3     private final String name;
 4  
 5     public CommandHelloFailure(String name) {
 6         super(Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey("ExampleGroup"))  //必須
 7                 .andCommandPropertiesDefaults(HystrixCommandProperties.Setter()
 8                         .withExecutionTimeoutInMilliseconds(1000))  //超時時間
 9                 .andThreadPoolKey(HystrixThreadPoolKey.Factory.asKey("ExampleGroup-pool"))
10                 .andThreadPoolPropertiesDefaults(HystrixThreadPoolProperties.Setter().withCoreSize(3)));
11  
12         this.name = name;
13     }
14  
15     @Override
16     protected String run() throws InterruptedException {
17         String theadName = this.getThreadPoolKey().name();
18         String cmdKey=this.getThreadPoolKey().name();
19         System.out.println("running begin , threadPool="+theadName+" cmdKey="+cmdKey+" name="+name);
20  
21         if("Exception".equals(name)) {
22             throw new RuntimeException("this command always fails");
23         }else if("Timeout".equals(name)){
24             TimeUnit.SECONDS.sleep(2);
25         }else if("Reject".equals(name)){
26             TimeUnit.MILLISECONDS.sleep(800);
27         }
28         System.out.println(" run end");
29  
30         return "Hello " + name + "!";
31     }
32  
33     @Override
34     protected String getFallback() {
35         StringBuilder sb = new StringBuilder("running fallback");
36         boolean isRejected = isResponseRejected();
37         boolean isException = isFailedExecution();
38         boolean isTimeout= isResponseTimedOut();
39         boolean isCircut = isCircuitBreakerOpen();
40  
41         sb.append(", isRejected:").append(isRejected);
42         sb.append(", isException:"+isException);
43         if(isException){
44             sb.append(" msg=").append(getExecutionException().getMessage());
45         }
46         sb.append(",  isTimeout: "+isTimeout);
47         sb.append(",  isCircut:"+isCircut);
48  
49         sb.append(", group:").append(this.getCommandGroup().name());
50         sb.append(", threadpool:").append(getThreadPoolKey().name());
51         System.out.println(sb.toString());
52  
53         String msg="Hello Failure " + name + "!";
54         return msg;
55     }
56 }

 

FAILURE 

測試由異常致使的fallback

1 @Test
2 public void expTest() {
3     assertEquals("Hello Failure Exception!", new CommandHelloFailure("Exception").execute());
4 }
5   

//控制檯輸出

running begin , threadPool=ExampleGroup-pool cmdKey=ExampleGroup-pool name=Exception
running fallback, isRejected:false, isException:true msg=this command always fails, isTimeout: false, isCircut:false, group:ExampleGroup, threadpool:ExampleGroup-pool

 

TIMEOUT

測試有超時致使的fallback

 

@Test
public void timeOutTest() {
    assertEquals("Hello Failure Timeout!", new CommandHelloFailure("Timeout").execute());
}
  

//控制檯輸出

running begin , threadPool=ExampleGroup-pool cmdKey=ExampleGroup-pool name=Timeout
running fallback, isRejected:false, isException:false, isTimeout: true, isCircut:false, group:ExampleGroup, threadpool:ExampleGroup-pool

 

THREAD_POOL_REJECTED

併發執行的任務數超過線程池和隊列之和會被reject，致使fallback

1 @Test
2 public void rejectTest() throws InterruptedException {
3     int count = 5;
4     while (count-- > 0){
5         new CommandHelloFailure("Reject").queue();
6         TimeUnit.MILLISECONDS.sleep(100);
7     }
8 }

//控制檯輸出

running begin , threadPool=ExampleGroup-pool cmdKey=ExampleGroup-pool name=Reject
running begin , threadPool=ExampleGroup-pool cmdKey=ExampleGroup-pool name=Reject
running begin , threadPool=ExampleGroup-pool cmdKey=ExampleGroup-pool name=Reject
running fallback, isRejected:true, isException:false, isTimeout: false, isCircut:false, group:ExampleGroup, threadpool:ExampleGroup-pool
running fallback, isRejected:true, isException:false, isTimeout: false, isCircut:false, group:ExampleGroup, threadpool:ExampleGroup-pool

SEMAPHORE_REJECTED  與 THREAD_POOL_REJECTED 相似，再也不演示

 

SHORT_CIRCUITED

在必定時間內，用戶請求超過必定的比例失敗時（timeout, failure, reject），斷路器就會打開；短路器打開後全部請求直接走fallback

參數設置

名稱	類型	含義	默認值
circuitBreakerEnabled	Boolean	是否啓用斷路器	true
circuitBreakerErrorThresholdPercentage	Integer	錯誤百分比，超過該值打開斷路器	50
circuitBreakerForceClosed	Boolean	強制斷路器打開	false
circuitBreakerForceOpen	Boolean	強制短路器關閉	false
circuitBreakerRequestVolumeThreshold	Integer	10s中內最少的請求量，大於該值，斷路器配置纔會生效	20
circuitBreakerSleepWindowInMilliseconds	Integer	短路器打開後多長時間嘗試關閉（Half open）	5s

 

 

 

 

 

 

 

通常配置以下：

1 Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey("ExampleGroup"))  //必須
2         .andCommandPropertiesDefaults(HystrixCommandProperties.Setter()
3                 .withExecutionTimeoutInMilliseconds(50)//超時時間
4                 .withCircuitBreakerRequestVolumeThreshold(5)
5                 .withCircuitBreakerSleepWindowInMilliseconds(1000)
6                 .withCircuitBreakerErrorThresholdPercentage(50))
7         .andThreadPoolKey(HystrixThreadPoolKey.Factory.asKey("ExampleGroup-pool"))  //可選,默認 使用 this.getClass().getSimpleName();
8         .andThreadPoolPropertiesDefaults(HystrixThreadPoolProperties.Setter().withCoreSize(4));

以上配置的含義是: 在10s內，若是請求在5個及以上，且有50%失敗的狀況下，開啓斷路器；斷路器開啓1000ms後嘗試關閉

短路器的工做機制，引用自官方文檔：

The precise way that the circuit opening and closing occurs is as follows:
Assuming the volume across a circuit meets a certain threshold (HystrixCommandProperties.circuitBreakerRequestVolumeThreshold())...
And assuming that the error percentage exceeds the threshold error percentage (HystrixCommandProperties.circuitBreakerErrorThresholdPercentage())...
Then the circuit-breaker transitions from CLOSED to OPEN.
While it is open, it short-circuits all requests made against that circuit-breaker.
After some amount of time (HystrixCommandProperties.circuitBreakerSleepWindowInMilliseconds()), the next single request is let through (this is the HALF-OPEN state). If the request fails, the circuit-breaker returns to the OPEN state for the duration of the sleep window. If the request succeeds, the circuit-breaker transitions to CLOSED and the logic in 1. takes over again.

Q3：fallback時咱們應該怎麼辦？

通常有如下幾種策略：

一、不實現getFallback方法：依賴調用失敗時直接拋出異常

二、實現getFallback方法，返回默認值：這是一種常見的策略

三、實現getFallback方法，走降級方案

此外，生產環境中，fallback時，通常須要打點記錄

請求合併

簡單來講，就是將一段時間內的屢次請求合併爲一次請求，經常使用於網絡IO中，能減小IO次數，缺點是增長平均延遲

如下是測試代碼主程序：

 1 public class CommandCollapserGetValueForKey extends HystrixCollapser<List<String>, String, Integer> {
 2  
 3     private final Integer key;
 4  
 5     public CommandCollapserGetValueForKey(Integer key) {
 6         super(Setter.withCollapserKey(HystrixCollapserKey.Factory.asKey("Collapser"))
 7                 .andCollapserPropertiesDefaults(HystrixCollapserProperties.Setter()
 8                         .withMaxRequestsInBatch(3)
 9                 .withTimerDelayInMilliseconds(10)));
10         this.key = key;
11     }
12  
13     @Override
14     public Integer getRequestArgument() {
15         return key;
16     }
17  
18     @Override
19     protected HystrixCommand<List<String>> createCommand(final Collection<CollapsedRequest<String, Integer>> requests) {
20         return new BatchCommand(requests);
21     }
22  
23     @Override
24     protected void mapResponseToRequests(List<String> batchResponse, Collection<CollapsedRequest<String, Integer>> requests) {
25         int count = 0;
26         for (CollapsedRequest<String, Integer> request : requests) {
27             request.setResponse(batchResponse.get(count++));
28         }
29     }
30  
31     private static final class BatchCommand extends HystrixCommand<List<String>> {
32         private final Collection<CollapsedRequest<String, Integer>> requests;
33  
34         private BatchCommand(Collection<CollapsedRequest<String, Integer>> requests) {
35             super(Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey("ExampleGroup"))
36                     .andCommandKey(HystrixCommandKey.Factory.asKey("GetValueForKey")));
37             this.requests = requests;
38         }
39  
40         @Override
41         protected List<String> run() {
42             System.out.println("BatchCommand run  "+requests.size());
43             ArrayList<String> response = new ArrayList<String>();
44             for (CollapsedRequest<String, Integer> request : requests) {
45                 // artificial response for each argument received in the batch
46                 response.add("ValueForKey: " + request.getArgument());
47             }
48             return response;
49         }
50     }
51 }
52   
53   
54 @Test
55 public void testCollapser() throws Exception {
56     HystrixRequestContext context = HystrixRequestContext.initializeContext();
57     try {
58         Future<String> f1 = new CommandCollapserGetValueForKey(1).queue();
59         Future<String> f2 = new CommandCollapserGetValueForKey(2).queue();
60         Future<String> f3 = new CommandCollapserGetValueForKey(3).queue();
61         Future<String> f4 = new CommandCollapserGetValueForKey(4).queue();
62  
63  
64         assertEquals("ValueForKey: 1", f1.get());
65         assertEquals("ValueForKey: 2", f2.get());
66         assertEquals("ValueForKey: 3", f3.get());
67         assertEquals("ValueForKey: 4", f4.get());
68  
69         // assert that the batch command 'GetValueForKey' was in fact
70         // executed and that it executed only once
71         assertEquals(2, HystrixRequestLog.getCurrentRequest().getAllExecutedCommands().size());
72         HystrixCommand<?> command = HystrixRequestLog.getCurrentRequest().getAllExecutedCommands().toArray(new HystrixCommand<?>[1])[0];
73         // assert the command is the one we're expecting
74         assertEquals("GetValueForKey", command.getCommandKey().name());
75         // confirm that it was a COLLAPSED command execution
76         assertTrue(command.getExecutionEvents().contains(HystrixEventType.COLLAPSED));
77         // and that it was successful
78         assertTrue(command.getExecutionEvents().contains(HystrixEventType.SUCCESS));
79     } finally {
80         context.shutdown();
81     }
82 }
83   
84 //控制輸出
85 BatchCommand run  3
86 BatchCommand run  1

執行流程：

 

使用該特性

一、必須繼承HystrixCollapser類，

二、實現如下方法：

getRequestArgument： 返回請求參數對象

createCommand ： 返回BatchCommand

mapResponseToRequests：實現Response和Request的映射

三、建立對應的BatchCommand類：批量請求的具體實現

 

參數配置：

名稱	類型	含義	默認值
maxRequestsInBatch	Integer	每一個批次最大的請求數，超過該值，建立新的batch請求	Integer.MAX_VALUE
timerDelayInMilliseconds	Integer	等待時間窗口，超過該值，建立新的batch請求	10ms
requestCacheEnabled	Boolean	是否啓用cache	true

 

 

 

 

 

 

通常配置以下

Setter.withCollapserKey(HystrixCollapserKey.Factory.asKey("Collapser"))
       .andCollapserPropertiesDefaults(HystrixCollapserProperties.Setter()
               .withMaxRequestsInBatch(3)
       .withTimerDelayInMilliseconds(5))；

 

請求cache

 1 public class CommandUsingRequestCache extends HystrixCommand<Boolean> {
 2     private final int value;
 3      
 4     public CommandUsingRequestCache(int value) {
 5         super(HystrixCommandGroupKey.Factory.asKey("ExampleGroup"));
 6         this.value = value;
 7     }
 8  
 9     @Override
10     public Boolean run() {
11         return value == 0 || value % 2 == 0;
12     }
13  
14    //使用cache功能，必須實現該方法
15     @Override
16     public String getCacheKey() {
17         return String.valueOf(value);
18     }
19 }
20   
21 @Test
22 public void testWithCacheHits() {
23     HystrixRequestContext context = HystrixRequestContext.initializeContext();
24     try {
25         CommandUsingRequestCache command2a = new CommandUsingRequestCache(2);
26         CommandUsingRequestCache command2b = new CommandUsingRequestCache(2);
27  
28         assertTrue(command2a.execute());
29         //第一次請求，沒有cache
30         assertFalse(command2a.isResponseFromCache());
31  
32         assertTrue(command2b.execute());
33         // 第二次請求，從cache中拿的結果
34         assertTrue(command2b.isResponseFromCache());
35     } finally {
36         context.shutdown();
37     }
38  
39     context = HystrixRequestContext.initializeContext();
40     try {
41         CommandUsingRequestCache command3b = new CommandUsingRequestCache(2);
42         assertTrue(command3b.execute());
43         // this is a new request context so this
44         //new了新的 request context後，以前的cache失效
45         assertFalse(command3b.isResponseFromCache());
46     } finally {
47         context.shutdown();
48     }
49 }

Hystrix Context

 Global Context

UserRequest Context

 

使用與監控

一、工程中使用

使用Hystrix很簡單，只須要添加相應依賴便可，以Maven爲例：

 

 1 <!-- hystrix 依賴 -->
 2 <dependency>
 3     <groupId>com.netflix.hystrix</groupId>
 4     <artifactId>hystrix-core</artifactId>
 5     <version>1.5.9</version>
 6 </dependency>
 7 <dependency>
 8     <groupId>com.netflix.hystrix</groupId>
 9     <artifactId>hystrix-metrics-event-stream</artifactId>
10     <version>1.5.9</version>
11 </dependency>

二、DashBoard使用 

web.xml中配置相應的Servlet

1 <servlet>
2           <display-name>HystrixMetricsStreamServlet</display-name>
3           <servlet-name>HystrixMetricsStreamServlet</servlet-name>
4           <servlet-class>com.netflix.hystrix.contrib.metrics.eventstream.HystrixMetricsStreamServlet</servlet-class>
5 </servlet>
6 <servlet-mapping>
7           <servlet-name>HystrixMetricsStreamServlet</servlet-name>
8           <url-pattern>/hystrix.stream</url-pattern>
9    </servlet-mapping>

下載附件中的war文件和jar文件到任意目錄，執行

java -jar jetty-runner-9.2.10.v20150310.jar --port 8410 hystrix-dashboard-1.5.1.war

而後在瀏覽器中打開：http://localhost:8410/  ，在輸入框中填寫 http://hostname:port/application/hystrix.stream， 點擊 Add Stream ,而後在點擊Monitor Stream， 看到以下圖：

每一個指標對應的含義：

通常來講： Thread-pool Rejections  和Failuress/Exception應該是0，Thread timeouts是個很小的值。

代碼結構

附件

一、啓動腳本 start.sh

二、hystrix-dashboard-1.5.1.war

三、jetty-runner-9.2.10.v20150310.jar