聊聊micrometer的HistogramGauges

序

本文主要研究一下micrometer的HistogramGauges

AutoConfiguration

針對springboot應用，配備有各類export的AutoConfiguration，詳見org.springframework.boot.actuate.autoconfigure.metrics.export包，2.0.1版本目前支持了以下類型的export：

atlas、datadog、ganglia、graphite、influx、jmx、newrelic、prometheus、properties、signalfx、simple、statsd、wavefront

這裏看下statsd及prometheus的AutoConfiguration

StatsdMetricsExportAutoConfiguration

spring-boot-actuator-autoconfigure-2.0.1.RELEASE-sources.jar!/org/springframework/boot/actuate/autoconfigure/metrics/export/statsd/StatsdMetricsExportAutoConfiguration.java

@Configuration
@AutoConfigureBefore({ CompositeMeterRegistryAutoConfiguration.class,
        SimpleMetricsExportAutoConfiguration.class })
@AutoConfigureAfter(MetricsAutoConfiguration.class)
@ConditionalOnBean(Clock.class)
@ConditionalOnClass(StatsdMeterRegistry.class)
@ConditionalOnProperty(prefix = "management.metrics.export.statsd", name = "enabled", havingValue = "true", matchIfMissing = true)
@EnableConfigurationProperties(StatsdProperties.class)
public class StatsdMetricsExportAutoConfiguration {

    @Bean
    @ConditionalOnMissingBean
    public StatsdConfig statsdConfig(StatsdProperties statsdProperties) {
        return new StatsdPropertiesConfigAdapter(statsdProperties);
    }

    @Bean
    @ConditionalOnMissingBean
    public StatsdMeterRegistry statsdMeterRegistry(StatsdConfig statsdConfig,
            Clock clock) {
        return new StatsdMeterRegistry(statsdConfig, clock);
    }

    @Bean
    public StatsdMetrics statsdMetrics() {
        return new StatsdMetrics();
    }

}

能夠看到，建立了StatsdMeterRegistry

PrometheusMetricsExportAutoConfiguration

spring-boot-actuator-autoconfigure-2.0.1.RELEASE-sources.jar!/org/springframework/boot/actuate/autoconfigure/metrics/export/prometheus/PrometheusMetricsExportAutoConfiguration.java

@Configuration
@AutoConfigureBefore({ CompositeMeterRegistryAutoConfiguration.class,
        SimpleMetricsExportAutoConfiguration.class })
@AutoConfigureAfter(MetricsAutoConfiguration.class)
@ConditionalOnBean(Clock.class)
@ConditionalOnClass(PrometheusMeterRegistry.class)
@ConditionalOnProperty(prefix = "management.metrics.export.prometheus", name = "enabled", havingValue = "true", matchIfMissing = true)
@EnableConfigurationProperties(PrometheusProperties.class)
public class PrometheusMetricsExportAutoConfiguration {

    @Bean
    @ConditionalOnMissingBean
    public PrometheusConfig prometheusConfig(PrometheusProperties prometheusProperties) {
        return new PrometheusPropertiesConfigAdapter(prometheusProperties);
    }

    @Bean
    @ConditionalOnMissingBean
    public PrometheusMeterRegistry prometheusMeterRegistry(
            PrometheusConfig prometheusConfig, CollectorRegistry collectorRegistry,
            Clock clock) {
        return new PrometheusMeterRegistry(prometheusConfig, collectorRegistry, clock);
    }

    @Bean
    @ConditionalOnMissingBean
    public CollectorRegistry collectorRegistry() {
        return new CollectorRegistry(true);
    }

    @ManagementContextConfiguration
    public static class PrometheusScrapeEndpointConfiguration {

        @Bean
        @ConditionalOnEnabledEndpoint
        @ConditionalOnMissingBean
        public PrometheusScrapeEndpoint prometheusEndpoint(
                CollectorRegistry collectorRegistry) {
            return new PrometheusScrapeEndpoint(collectorRegistry);
        }

    }

}

能夠看到建立了PrometheusMeterRegistry

Timer.register

micrometer-core-1.0.3-sources.jar!/io/micrometer/core/instrument/Timer.java

/**
         * Add the timer to a single registry, or return an existing timer in that registry. The returned
         * timer will be unique for each registry, but each registry is guaranteed to only create one timer
         * for the same combination of name and tags.
         *
         * @param registry A registry to add the timer to, if it doesn't already exist.
         * @return A new or existing timer.
         */
        public Timer register(MeterRegistry registry) {
            // the base unit for a timer will be determined by the monitoring system implementation
            return registry.timer(new Meter.Id(name, tags, null, description, Type.TIMER), distributionConfigBuilder.build(),
                    pauseDetector == null ? registry.config().pauseDetector() : pauseDetector);
        }

能夠看到該register委託給了registry.timer方法

MeterRegistry

micrometer-core-1.0.3-sources.jar!/io/micrometer/core/instrument/MeterRegistry.java

/**
     * Only used by {@link Timer#builder(String)}.
     *
     * @param id                          The identifier for this timer.
     * @param distributionStatisticConfig Configuration that governs how distribution statistics are computed.
     * @return A new or existing timer.
     */
    Timer timer(Meter.Id id, DistributionStatisticConfig distributionStatisticConfig, PauseDetector pauseDetectorOverride) {
        return registerMeterIfNecessary(Timer.class, id, distributionStatisticConfig, (id2, filteredConfig) -> {
            Meter.Id withUnit = id2.withBaseUnit(getBaseTimeUnitStr());
            return newTimer(withUnit, filteredConfig.merge(defaultHistogramConfig()), pauseDetectorOverride);
        }, NoopTimer::new);
    }

    /**
     * Build a new timer to be added to the registry. This is guaranteed to only be called if the timer doesn't already exist.
     *
     * @param id                          The id that uniquely identifies the timer.
     * @param distributionStatisticConfig Configuration for published distribution statistics.
     * @param pauseDetector               The pause detector to use for coordinated omission compensation.
     * @return A new timer.
     */
    protected abstract Timer newTimer(Meter.Id id, DistributionStatisticConfig distributionStatisticConfig, PauseDetector pauseDetector);

這裏有調用了newTimer抽象方法

StatsdMeterRegistry.newTimer

micrometer-registry-statsd-1.0.3-sources.jar!/io/micrometer/statsd/StatsdMeterRegistry.java

@SuppressWarnings("ConstantConditions")
    @Override
    protected Timer newTimer(Meter.Id id, DistributionStatisticConfig distributionStatisticConfig, PauseDetector
            pauseDetector) {
        Timer timer = new StatsdTimer(id, lineBuilder(id), publisher, clock, distributionStatisticConfig, pauseDetector, getBaseTimeUnit(),
                statsdConfig.step().toMillis());
        HistogramGauges.registerWithCommonFormat(timer, this);
        return timer;
    }

能夠看到newTimer操做裏頭調用了HistogramGauges.registerWithCommonFormat(timer, this);

HistogramGauges.registerWithCommonFormat

micrometer-core-1.0.3-sources.jar!/io/micrometer/core/instrument/distribution/HistogramGauges.java

/**
     * Register a set of gauges for percentiles and histogram buckets that follow a common format when
     * the monitoring system doesn't have an opinion about the structure of this data.
     */
    public static HistogramGauges registerWithCommonFormat(Timer timer, MeterRegistry registry) {
        Meter.Id id = timer.getId();
        return HistogramGauges.register(timer, registry,
                percentile -> id.getName() + ".percentile",
                percentile -> Tags.concat(id.getTags(), "phi", DoubleFormat.decimalOrNan(percentile.percentile())),
                percentile -> percentile.value(timer.baseTimeUnit()),
                bucket -> id.getName() + ".histogram",
                bucket -> Tags.concat(id.getTags(), "le", DoubleFormat.decimalOrWhole(bucket.bucket(timer.baseTimeUnit()))));
    }

能夠看到這裏使用HistogramGauges進行註冊，percentileName的名稱爲id.getName() + ".percentile"，bucketName的名稱爲id.getName() + ".histogram"

HistogramGauges

micrometer-core-1.0.3-sources.jar!/io/micrometer/core/instrument/distribution/HistogramGauges.java

private HistogramGauges(HistogramSupport meter, MeterRegistry registry,
                            Function<ValueAtPercentile, String> percentileName,
                            Function<ValueAtPercentile, Iterable<Tag>> percentileTags,
                            Function<ValueAtPercentile, Double> percentileValue,
                            Function<CountAtBucket, String> bucketName,
                            Function<CountAtBucket, Iterable<Tag>> bucketTags) {
        this.meter = meter;

        HistogramSnapshot initialSnapshot = meter.takeSnapshot();
        this.snapshot = initialSnapshot;

        ValueAtPercentile[] valueAtPercentiles = initialSnapshot.percentileValues();
        CountAtBucket[] countAtBuckets = initialSnapshot.histogramCounts();

        this.totalGauges = valueAtPercentiles.length + countAtBuckets.length;

        // set to zero initially, so the first polling of one of the gauges on each publish cycle results in a
        // new snapshot
        this.polledGaugesLatch = new CountDownLatch(0);

        for (int i = 0; i < valueAtPercentiles.length; i++) {
            final int index = i;

            ToDoubleFunction<HistogramSupport> percentileValueFunction = m -> {
                snapshotIfNecessary();
                polledGaugesLatch.countDown();
                return percentileValue.apply(snapshot.percentileValues()[index]);
            };

            Gauge.builder(percentileName.apply(valueAtPercentiles[i]), meter, percentileValueFunction)
                    .tags(percentileTags.apply(valueAtPercentiles[i]))
                    .register(registry);
        }

        for (int i = 0; i < countAtBuckets.length; i++) {
            final int index = i;

            ToDoubleFunction<HistogramSupport> bucketCountFunction = m -> {
                snapshotIfNecessary();
                polledGaugesLatch.countDown();
                return snapshot.histogramCounts()[index].count();
            };

            Gauge.builder(bucketName.apply(countAtBuckets[i]), meter, bucketCountFunction)
                    .tags(bucketTags.apply(countAtBuckets[i]))
                    .register(registry);
        }
    }

能夠看到這裏針對HistogramSnapshot取了percentileValues註冊了Gauge，而後針對HistogramSnapshot的CountAtBucket[]註冊了對應的Gauge

實例

SimpleMeterRegistry simpleMeterRegistry = new SimpleMeterRegistry();
    @Test
    public void testHistogramGauges() throws InterruptedException {
        Timer timer = Timer.builder("api-cost")
                .publishPercentileHistogram()
                .publishPercentiles(0.95,0.99)
                .register(simpleMeterRegistry);

        IntStream.rangeClosed(1,1000)
                .forEach(i -> {
                    timer.record(Duration.ofMillis(ThreadLocalRandom.current().nextInt(200)));
                    simpleMeterRegistry.getMeters()
                            .stream()
                            .forEach(m -> {
                                System.out.println(m.getId() + "-->" + m.measure());
                            });
                });
        TimeUnit.MINUTES.sleep(5);
    }

輸出實例

MeterId{name='api-cost.percentile', tags=[ImmutableTag{key='phi', value='0.95'}]}-->[Measurement{statistic='VALUE', value=0.192905216}]
MeterId{name='api-cost.percentile', tags=[ImmutableTag{key='phi', value='0.99'}]}-->[Measurement{statistic='VALUE', value=0.201293824}]
MeterId{name='api-cost', tags=[]}-->[Measurement{statistic='COUNT', value=999.0}, Measurement{statistic='TOTAL_TIME', value=97.158}, Measurement{statistic='MAX', value=0.199}]
MeterId{name='api-cost.percentile', tags=[ImmutableTag{key='phi', value='0.95'}]}-->[Measurement{statistic='VALUE', value=0.192905216}]
MeterId{name='api-cost.percentile', tags=[ImmutableTag{key='phi', value='0.99'}]}-->[Measurement{statistic='VALUE', value=0.201293824}]
MeterId{name='api-cost', tags=[]}-->[Measurement{statistic='COUNT', value=1000.0}, Measurement{statistic='TOTAL_TIME', value=97.348}, Measurement{statistic='MAX', value=0.199}]

小結

目前只有Prometheus和Atlas支持Percentile histograms，不過micrometer在client端簡單支持了下percentile，不過不像server端支持那麼靈活，不能跨tag進行聚合，目前是把tag做爲meter id的一部分，一塊兒上報。針對qps的計算，能夠使用Timer類型來計量，而後經過percentile指標，根據時間間隔進行group來統計。

doc

• 13. Histograms and percentiles