聊聊sentinel的DegradeSlot

序

本文主要研究一下sentinel的DegradeSlot

DegradeSlot

com/alibaba/csp/sentinel/slots/block/degrade/DegradeSlot.java

public class DegradeSlot extends AbstractLinkedProcessorSlot<DefaultNode> {

    @Override
    public void entry(Context context, ResourceWrapper resourceWrapper, DefaultNode node, int count, Object... args)
        throws Throwable {
        DegradeRuleManager.checkDegrade(resourceWrapper, context, node, count);
        fireEntry(context, resourceWrapper, node, count, args);
    }

    @Override
    public void exit(Context context, ResourceWrapper resourceWrapper, int count, Object... args) {
        fireExit(context, resourceWrapper, count, args);
    }
}

• 調用DegradeRuleManager.checkDegrade進行降級規則檢測

DegradeRuleManager

com/alibaba/csp/sentinel/slots/block/degrade/DegradeRuleManager.java

public class DegradeRuleManager {

    private static volatile Map<String, List<DegradeRule>> degradeRules
        = new ConcurrentHashMap<String, List<DegradeRule>>();

    final static RulePropertyListener listener = new RulePropertyListener();
    private static SentinelProperty<List<DegradeRule>> currentProperty
        = new DynamicSentinelProperty<List<DegradeRule>>();

    static {
        currentProperty.addListener(listener);
    }

    /**
     * Listen to the {@link SentinelProperty} for {@link DegradeRule}s. The property is the source
     * of {@link DegradeRule}s. Degrade rules can also be set by {@link #loadRules(List)} directly.
     *
     * @param property the property to listen.
     */
    public static void register2Property(SentinelProperty<List<DegradeRule>> property) {
        synchronized (listener) {
            currentProperty.removeListener(listener);
            property.addListener(listener);
            currentProperty = property;
        }
    }

    public static void checkDegrade(ResourceWrapper resource, Context context, DefaultNode node, int count)
        throws BlockException {
        if (degradeRules == null) {
            return;
        }

        List<DegradeRule> rules = degradeRules.get(resource.getName());
        if (rules == null) {
            return;
        }

        for (DegradeRule rule : rules) {
            if (!rule.passCheck(context, node, count)) {
                throw new DegradeException(rule.getLimitApp());
            }
        }
    }

    //......
}

• checkDegrade根據資源名稱獲取對應的降級規則，而後挨個遍歷檢查

DegradeRule

com/alibaba/csp/sentinel/slots/block/degrade/DegradeRule.java

/**
 * <p>
 * Degrade is used when the resources are in an unstable state, these resources
 * will be degraded within the next defined time window. There are two ways to
 * measure whether a resource is stable or not:
 * </p>
 * <ul>
 * <li>
 * Average response time ({@code DEGRADE_GRADE_RT}): When
 * the average RT exceeds the threshold ('count' in 'DegradeRule', in milliseconds), the
 * resource enters a quasi-degraded state. If the RT of next coming 5
 * requests still exceed this threshold, this resource will be downgraded, which
 * means that in the next time window (defined in 'timeWindow', in seconds) all the
 * access to this resource will be blocked.
 * </li>
 * <li>
 * Exception ratio: When the ratio of exception count per second and the
 * success qps exceeds the threshold, access to the resource will be blocked in
 * the coming window.
 * </li>
 * </ul>
 *
 * @author jialiang.linjl
 */
public class DegradeRule extends AbstractRule {

    private static final int RT_MAX_EXCEED_N = 5;

    private static ScheduledExecutorService pool = Executors.newScheduledThreadPool(
        Runtime.getRuntime().availableProcessors(), new NamedThreadFactory("sentinel-degrade-reset-task", true));

    /**
     * RT threshold or exception ratio threshold count.
     */
    private double count;

    /**
     * Degrade recover timeout (in seconds) when degradation occurs.
     */
    private int timeWindow;

    /**
     * Degrade strategy (0: average RT, 1: exception ratio).
     */
    private int grade = RuleConstant.DEGRADE_GRADE_RT;

    private volatile boolean cut = false;

    public int getGrade() {
        return grade;
    }

    public void setGrade(int grade) {
        this.grade = grade;
    }

    private AtomicLong passCount = new AtomicLong(0);

    private final Object lock = new Object();

    //......

    @Override
    public boolean passCheck(Context context, DefaultNode node, int acquireCount, Object... args) {

        if (cut) {
            return false;
        }

        ClusterNode clusterNode = ClusterBuilderSlot.getClusterNode(this.getResource());
        if (clusterNode == null) {
            return true;
        }

        if (grade == RuleConstant.DEGRADE_GRADE_RT) {
            double rt = clusterNode.avgRt();
            if (rt < this.count) {
                passCount.set(0);
                return true;
            }

            // Sentinel will degrade the service only if count exceeds.
            if (passCount.incrementAndGet() < RT_MAX_EXCEED_N) {
                return true;
            }
        } else {
            double exception = clusterNode.exceptionQps();
            double success = clusterNode.successQps();
            long total = clusterNode.totalQps();
            // if total qps less than RT_MAX_EXCEED_N, pass.
            if (total < RT_MAX_EXCEED_N) {
                return true;
            }

            if (success == 0) {
                return exception < RT_MAX_EXCEED_N;
            }

            if (exception / (success + exception) < count) {
                return true;
            }
        }

        synchronized (lock) {
            if (!cut) {
                // Automatically degrade.
                cut = true;
                ResetTask resetTask = new ResetTask(this);
                pool.schedule(resetTask, timeWindow, TimeUnit.SECONDS);
            }

            return false;
        }
    }

    //......

    private static final class ResetTask implements Runnable {

        private DegradeRule rule;

        ResetTask(DegradeRule rule) {
            this.rule = rule;
        }

        @Override
        public void run() {
            rule.getPassCount().set(0);
            rule.setCut(false);
        }
    }
}

• 這個passCheck根據平均響應時間以及異常個數等進行降級判斷
• 觸發降級時標誌cut爲true，而後啓動一個ResetTask，在指定時間窗口以後重置cut爲false並清空passCount計數

小結

sentinel的DegradeSlot主要依據平均響應時間以及異常次數來判斷，進入降級模式時啓動定時任務在指定時間窗口重置相關計數，恢復到正常模式。

doc

• DegradeSlot