Alertmanager經過命令行標誌和配置文件進行配置。 雖然命令行標誌配置了不可變的系統參數,但配置文件定義了禁止規則,通知路由和通知接收器。html
可視化編輯器能夠幫助構建路由樹。node
要查看全部可用的命令行標誌,請運行alertmanager -h
。mysql
Alertmanager能夠在運行時從新加載其配置。 若是新配置格式不正確,則不會應用更改並記錄錯誤。 經過向進程發送SIGHUP
或向/-/reload
端點發送HTTP POST請求來觸發配置從新加載。git
指定要加載的配置文件,使用--config.file
標誌github
./alertmanager --config.file=simple.yml
複製代碼
該文件以YAML格式寫入,由下面描述的方案定義。括號表示參數是可選的。對於非列表參數,該值設置爲指定的默認值。web
通用佔位符定義以下:正則表達式
<duration>
:與正則表達式匹配的持續時間[0-9]+(ms|[smhdwy])
<labelname>
:與正則表達式匹配的字符串[a-zA-Z _][a-zA-Z0-9 _]*
<labelvalue>
:一串unicode字符<filepath>
:當前工做目錄中的有效路徑<boolean>
:一個能夠取值爲true
或false
的布爾值<string>
:常規字符串<secret>
:一個祕密的常規字符串,例如密碼<tmpl_string>
:在使用前進行模板擴展的字符串<tmpl_secret>
:在使用以前進行模板擴展的字符串,它是一個祕密 其餘佔位符是單獨指定的。能夠在此處找到有效的示例文件。sql
全局配置指定在全部其餘配置上下文中有效的參數。它們還可用做其餘配置節的默認值。api
global:
# ResolveTimeout is the time after which an alert is declared resolved
# if it has not been updated.
[ resolve_timeout: <duration> | default = 5m ]
# The default SMTP From header field.
[ smtp_from: <tmpl_string> ]
# The default SMTP smarthost used for sending emails, including port number.
# Port number usually is 25, or 587 for SMTP over TLS (sometimes referred to as STARTTLS).
# Example: smtp.example.org:587
[ smtp_smarthost: <string> ]
# The default hostname to identify to the SMTP server.
[ smtp_hello: <string> | default = "localhost" ]
[ smtp_auth_username: <string> ]
# SMTP Auth using LOGIN and PLAIN.
[ smtp_auth_password: <secret> ]
# SMTP Auth using PLAIN.
[ smtp_auth_identity: <string> ]
# SMTP Auth using CRAM-MD5.
[ smtp_auth_secret: <secret> ]
# The default SMTP TLS requirement.
[ smtp_require_tls: <bool> | default = true ]
# The API URL to use for Slack notifications.
[ slack_api_url: <secret> ]
[ victorops_api_key: <secret> ]
[ victorops_api_url: <string> | default = "https://alert.victorops.com/integrations/generic/20131114/alert/" ]
[ pagerduty_url: <string> | default = "https://events.pagerduty.com/v2/enqueue" ]
[ opsgenie_api_key: <secret> ]
[ opsgenie_api_url: <string> | default = "https://api.opsgenie.com/" ]
[ hipchat_api_url: <string> | default = "https://api.hipchat.com/" ]
[ hipchat_auth_token: <secret> ]
[ wechat_api_url: <string> | default = "https://qyapi.weixin.qq.com/cgi-bin/" ]
[ wechat_api_secret: <secret> ]
[ wechat_api_corp_id: <string> ]
# The default HTTP client configuration
[ http_config: <http_config> ]
# Files from which custom notification template definitions are read.
# The last component may use a wildcard matcher, e.g. 'templates/*.tmpl'.
templates:
[ - <filepath> ... ]
# The root node of the routing tree.
route: <route>
# A list of notification receivers.
receivers:
- <receiver> ...
# A list of inhibition rules.
inhibit_rules:
[ - <inhibit_rule> ... ]
複製代碼
<route>
路由塊定義路由樹中的節點及其子節點。 若是未設置,則其可選配置參數將從其父節點繼承。bash
每一個警報都在配置的頂級路由中進入路由樹,該路由必須匹配全部警報(即沒有任何已配置的匹配器)。 而後它遍歷子節點。 若是將continue
設置爲false,則在第一個匹配的子項後中止。 若是匹配節點上的continue
爲true,則警報將繼續與後續兄弟節點匹配。 若是警報與節點的任何子節點都不匹配(沒有匹配的子節點,或者不存在),則根據當前節點的配置參數處理警報。
[ receiver: <string> ]
# The labels by which incoming alerts are grouped together. For example,
# multiple alerts coming in for cluster=A and alertname=LatencyHigh would
# be batched into a single group.
#
# To aggregate by all possible labels use the special value '...' as the sole label name, for example:
# group_by: ['...']
# This effectively disables aggregation entirely, passing through all
# alerts as-is. This is unlikely to be what you want, unless you have
# a very low alert volume or your upstream notification system performs
# its own grouping.
[ group_by: '[' <labelname>, ... ']' ]
# Whether an alert should continue matching subsequent sibling nodes.
[ continue: <boolean> | default = false ]
# A set of equality matchers an alert has to fulfill to match the node.
match:
[ <labelname>: <labelvalue>, ... ]
# A set of regex-matchers an alert has to fulfill to match the node.
match_re:
[ <labelname>: <regex>, ... ]
# How long to initially wait to send a notification for a group
# of alerts. Allows to wait for an inhibiting alert to arrive or collect
# more initial alerts for the same group. (Usually ~0s to few minutes.)
[ group_wait: <duration> | default = 30s ]
# How long to wait before sending a notification about new alerts that
# are added to a group of alerts for which an initial notification has
# already been sent. (Usually ~5m or more.)
[ group_interval: <duration> | default = 5m ]
# How long to wait before sending a notification again if it has already
# been sent successfully for an alert. (Usually ~3h or more).
[ repeat_interval: <duration> | default = 4h ]
# Zero or more child routes.
routes:
[ - <route> ... ]
複製代碼
例子:
# The root route with all parameters, which are inherited by the child
# routes if they are not overwritten.
route:
receiver: 'default-receiver'
group_wait: 30s
group_interval: 5m
repeat_interval: 4h
group_by: [cluster, alertname]
# All alerts that do not match the following child routes
# will remain at the root node and be dispatched to 'default-receiver'.
routes:
# All alerts with service=mysql or service=cassandra
# are dispatched to the database pager.
- receiver: 'database-pager'
group_wait: 10s
match_re:
service: mysql|cassandra
# All alerts with the team=frontend label match this sub-route.
# They are grouped by product and environment rather than cluster
# and alertname.
- receiver: 'frontend-pager'
group_by: [product, environment]
match:
team: frontend
複製代碼
<inhibit_rule>
當存在與另外一組匹配器匹配的警報(源)時,禁止規則將匹配一組匹配器的警報(目標)靜音。 目標和源警報必須具備相同列表中標籤名稱的equal
標籤值。
從語義上講,缺乏標籤和具備空值的標籤是equal
的。 所以,若是源和目標警報中都缺乏全部相同的標籤名稱,則禁用規則將適用。
爲了防止警報抑制自身,禁止規則將永遠不會禁止與規則的目標和源側匹配的警報。 可是,咱們建議以警報永遠不會匹配雙方的方式選擇目標和源匹配器。 理由更容易,而且不會觸發這種特殊狀況。
# Matchers that have to be fulfilled in the alerts to be muted.
target_match:
[ <labelname>: <labelvalue>, ... ]
target_match_re:
[ <labelname>: <regex>, ... ]
# Matchers for which one or more alerts have to exist for the
# inhibition to take effect.
source_match:
[ <labelname>: <labelvalue>, ... ]
source_match_re:
[ <labelname>: <regex>, ... ]
# Labels that must have an equal value in the source and target
# alert for the inhibition to take effect.
[ equal: '[' <labelname>, ... ']' ]
複製代碼
<http_config>
http_config
容許配置接收器用於與基於HTTP的API服務通訊的HTTP客戶端。
# Note that `basic_auth`, `bearer_token` and `bearer_token_file` options are
# mutually exclusive.
# Sets the `Authorization` header with the configured username and password.
# password and password_file are mutually exclusive.
basic_auth:
[ username: <string> ]
[ password: <secret> ]
[ password_file: <string> ]
# Sets the `Authorization` header with the configured bearer token.
[ bearer_token: <secret> ]
# Sets the `Authorization` header with the bearer token read from the configured file.
[ bearer_token_file: <filepath> ]
# Configures the TLS settings.
tls_config:
[ <tls_config> ]
# Optional proxy URL.
[ proxy_url: <string> ]
複製代碼
<tls_config>
tls_config
容許配置TLS鏈接。
# CA certificate to validate the server certificate with.
[ ca_file: <filepath> ]
# Certificate and key files for client cert authentication to the server.
[ cert_file: <filepath> ]
[ key_file: <filepath> ]
# ServerName extension to indicate the name of the server.
# http://tools.ietf.org/html/rfc4366#section-3.1
[ server_name: <string> ]
# Disable validation of the server certificate.
[ insecure_skip_verify: <boolean> | default = false]
複製代碼
<receiver>
Receiver是一個或多個通知集成的命名配置。
咱們沒有主動添加新的接收器,咱們建議經過webhook接收器實現自定義通知集成。
# The unique name of the receiver.
name: <string>
# Configurations for several notification integrations.
email_configs:
[ - <email_config>, ... ]
hipchat_configs:
[ - <hipchat_config>, ... ]
pagerduty_configs:
[ - <pagerduty_config>, ... ]
pushover_configs:
[ - <pushover_config>, ... ]
slack_configs:
[ - <slack_config>, ... ]
opsgenie_configs:
[ - <opsgenie_config>, ... ]
webhook_configs:
[ - <webhook_config>, ... ]
victorops_configs:
[ - <victorops_config>, ... ]
wechat_configs:
[ - <wechat_config>, ... ]
複製代碼
<email_config>
# Whether or not to notify about resolved alerts.
[ send_resolved: <boolean> | default = false ]
# The email address to send notifications to.
to: <tmpl_string>
# The sender address.
[ from: <tmpl_string> | default = global.smtp_from ]
# The SMTP host through which emails are sent.
[ smarthost: <string> | default = global.smtp_smarthost ]
# The hostname to identify to the SMTP server.
[ hello: <string> | default = global.smtp_hello ]
# SMTP authentication information.
[ auth_username: <string> | default = global.smtp_auth_username ]
[ auth_password: <secret> | default = global.smtp_auth_password ]
[ auth_secret: <secret> | default = global.smtp_auth_secret ]
[ auth_identity: <string> | default = global.smtp_auth_identity ]
# The SMTP TLS requirement.
[ require_tls: <bool> | default = global.smtp_require_tls ]
# TLS configuration.
tls_config:
[ <tls_config> ]
# The HTML body of the email notification.
[ html: <tmpl_string> | default = '{{ template "email.default.html" . }}' ]
# The text body of the email notification.
[ text: <tmpl_string> ]
# Further headers email header key/value pairs. Overrides any headers
# previously set by the notification implementation.
[ headers: { <string>: <tmpl_string>, ... } ]
複製代碼
<hipchat_config>
HipChat通知使用Build Your Own集成。
# Whether or not to notify about resolved alerts.
[ send_resolved: <boolean> | default = false ]
# The HipChat Room ID.
room_id: <tmpl_string>
# The auth token.
[ auth_token: <secret> | default = global.hipchat_auth_token ]
# The URL to send API requests to.
[ api_url: <string> | default = global.hipchat_api_url ]
# See https://www.hipchat.com/docs/apiv2/method/send_room_notification
# A label to be shown in addition to the sender's name.
[ from: <tmpl_string> | default = '{{ template "hipchat.default.from" . }}' ]
# The message body.
[ message: <tmpl_string> | default = '{{ template "hipchat.default.message" . }}' ]
# Whether this message should trigger a user notification.
[ notify: <boolean> | default = false ]
# Determines how the message is treated by the alertmanager and rendered inside HipChat. Valid values are 'text' and 'html'.
[ message_format: <string> | default = 'text' ]
# Background color for message.
[ color: <tmpl_string> | default = '{{ if eq .Status "firing" }}red{{ else }}green{{ end }}' ]
# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]
複製代碼
<pagerduty_config>
PagerDuty通知經過PagerDuty API發送。 PagerDuty提供了有關如何在此集成的文檔。
# Whether or not to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]
# The following two options are mutually exclusive.
# The PagerDuty integration key (when using PagerDuty integration type `Events API v2`).
routing_key: <tmpl_secret>
# The PagerDuty integration key (when using PagerDuty integration type `Prometheus`).
service_key: <tmpl_secret>
# The URL to send API requests to
[ url: <string> | default = global.pagerduty_url ]
# The client identification of the Alertmanager.
[ client: <tmpl_string> | default = '{{ template "pagerduty.default.client" . }}' ]
# A backlink to the sender of the notification.
[ client_url: <tmpl_string> | default = '{{ template "pagerduty.default.clientURL" . }}' ]
# A description of the incident.
[ description: <tmpl_string> | default = '{{ template "pagerduty.default.description" .}}' ]
# Severity of the incident.
[ severity: <tmpl_string> | default = 'error' ]
# A set of arbitrary key/value pairs that provide further detail
# about the incident.
[ details: { <string>: <tmpl_string>, ... } | default = {
firing: '{{ template "pagerduty.default.instances" .Alerts.Firing }}'
resolved: '{{ template "pagerduty.default.instances" .Alerts.Resolved }}'
num_firing: '{{ .Alerts.Firing | len }}'
num_resolved: '{{ .Alerts.Resolved | len }}'
} ]
# Images to attach to the incident.
images:
[ <image_config> ... ]
# Links to attach to the incident.
links:
[ <link_config> ... ]
# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]
複製代碼
<image_config>
這些字段記錄在PagerDuty API文檔中。
source: <tmpl_string>
alt: <tmpl_string>
text: <tmpl_string>
複製代碼
<link_config>
這些字段記錄在PagerDuty API文檔中。
href: <tmpl_string>
text: <tmpl_string>
複製代碼
<pushover_config>
推送通知經過Pushover API發送。
# Whether or not to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]
# The recipient user’s user key.
user_key: <secret>
# Your registered application’s API token, see https://pushover.net/apps
token: <secret>
# Notification title.
[ title: <tmpl_string> | default = '{{ template "pushover.default.title" . }}' ]
# Notification message.
[ message: <tmpl_string> | default = '{{ template "pushover.default.message" . }}' ]
# A supplementary URL shown alongside the message.
[ url: <tmpl_string> | default = '{{ template "pushover.default.url" . }}' ]
# Priority, see https://pushover.net/api#priority
[ priority: <tmpl_string> | default = '{{ if eq .Status "firing" }}2{{ else }}0{{ end }}' ]
# How often the Pushover servers will send the same notification to the user.
# Must be at least 30 seconds.
[ retry: <duration> | default = 1m ]
# How long your notification will continue to be retried for, unless the user
# acknowledges the notification.
[ expire: <duration> | default = 1h ]
# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]
複製代碼
<slack_config>
Slack通知經過Slack webhooks發送。 通知包含附件。
# Whether or not to notify about resolved alerts.
[ send_resolved: <boolean> | default = false ]
# The Slack webhook URL.
[ api_url: <secret> | default = global.slack_api_url ]
# The channel or user to send notifications to.
channel: <tmpl_string>
# API request data as defined by the Slack webhook API.
[ icon_emoji: <tmpl_string> ]
[ icon_url: <tmpl_string> ]
[ link_names: <boolean> | default = false ]
[ username: <tmpl_string> | default = '{{ template "slack.default.username" . }}' ]
# The following parameters define the attachment.
actions:
[ <action_config> ... ]
[ callback_id: <tmpl_string> | default = '{{ template "slack.default.callbackid" . }}' ]
[ color: <tmpl_string> | default = '{{ if eq .Status "firing" }}danger{{ else }}good{{ end }}' ]
[ fallback: <tmpl_string> | default = '{{ template "slack.default.fallback" . }}' ]
fields:
[ <field_config> ... ]
[ footer: <tmpl_string> | default = '{{ template "slack.default.footer" . }}' ]
[ pretext: <tmpl_string> | default = '{{ template "slack.default.pretext" . }}' ]
[ short_fields: <boolean> | default = false ]
[ text: <tmpl_string> | default = '{{ template "slack.default.text" . }}' ]
[ title: <tmpl_string> | default = '{{ template "slack.default.title" . }}' ]
[ title_link: <tmpl_string> | default = '{{ template "slack.default.titlelink" . }}' ]
[ image_url: <tmpl_string> ]
[ thumb_url: <tmpl_string> ]
# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]
複製代碼
<action_config>
這些字段記錄在Slack API文檔中。
type: <tmpl_string>
text: <tmpl_string>
url: <tmpl_string>
[ style: <tmpl_string> [ default = '' ]
複製代碼
<field_config>
這些字段記錄在Slack API文檔中。
title: <tmpl_string>
value: <tmpl_string>
[ short: <boolean> | default = slack_config.short_fields ]
複製代碼
<opsgenie_config>
OpsGenie通知經過OpsGenie API發送。
# Whether or not to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]
# The API key to use when talking to the OpsGenie API.
[ api_key: <secret> | default = global.opsgenie_api_key ]
# The host to send OpsGenie API requests to.
[ api_url: <string> | default = global.opsgenie_api_url ]
# Alert text limited to 130 characters.
[ message: <tmpl_string> ]
# A description of the incident.
[ description: <tmpl_string> | default = '{{ template "opsgenie.default.description" . }}' ]
# A backlink to the sender of the notification.
[ source: <tmpl_string> | default = '{{ template "opsgenie.default.source" . }}' ]
# A set of arbitrary key/value pairs that provide further detail
# about the incident.
[ details: { <string>: <tmpl_string>, ... } ]
# Comma separated list of team responsible for notifications.
[ teams: <tmpl_string> ]
# Comma separated list of tags attached to the notifications.
[ tags: <tmpl_string> ]
# Additional alert note.
[ note: <tmpl_string> ]
# Priority level of alert. Possible values are P1, P2, P3, P4, and P5.
[ priority: <tmpl_string> ]
# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]
複製代碼
<victorcops_config>
VictorOps通知經過VictorOps API發送出去
# Whether or not to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]
# The API key to use when talking to the VictorOps API.
[ api_key: <secret> | default = global.victorops_api_key ]
# The VictorOps API URL.
[ api_url: <string> | default = global.victorops_api_url ]
# A key used to map the alert to a team.
routing_key: <tmpl_string>
# Describes the behavior of the alert (CRITICAL, WARNING, INFO).
[ message_type: <tmpl_string> | default = 'CRITICAL' ]
# Contains summary of the alerted problem.
[ entity_display_name: <tmpl_string> | default = '{{ template "victorops.default.entity_display_name" . }}' ]
# Contains long explanation of the alerted problem.
[ state_message: <tmpl_string> | default = '{{ template "victorops.default.state_message" . }}' ]
# The monitoring tool the state message is from.
[ monitoring_tool: <tmpl_string> | default = '{{ template "victorops.default.monitoring_tool" . }}' ]
# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]
複製代碼
<webhook_config>
webhook接收器容許配置通用接收器。
# Whether or not to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]
# The endpoint to send HTTP POST requests to.
url: <string>
# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]
複製代碼
Alertmanager將如下列JSON格式將HTTP POST請求發送到配置的端點:
{
"version": "4",
"groupKey": <string>, // key identifying the group of alerts (e.g. to deduplicate)
"status": "<resolved|firing>",
"receiver": <string>,
"groupLabels": <object>,
"commonLabels": <object>,
"commonAnnotations": <object>,
"externalURL": <string>, // backlink to the Alertmanager.
"alerts": [
{
"status": "<resolved|firing>",
"labels": <object>,
"annotations": <object>,
"startsAt": "<rfc3339>",
"endsAt": "<rfc3339>",
"generatorURL": <string> // identifies the entity that caused the alert
},
...
]
}
複製代碼
有一個與此功能集成的列表。
<wechat_config>
微信通知經過微信API發送。
# Whether or not to notify about resolved alerts.
[ send_resolved: <boolean> | default = false ]
# The API key to use when talking to the WeChat API.
[ api_secret: <secret> | default = global.wechat_api_secret ]
# The WeChat API URL.
[ api_url: <string> | default = global.wechat_api_url ]
# The corp id for authentication.
[ corp_id: <string> | default = global.wechat_api_corp_id ]
# API request data as defined by the WeChat API.
[ message: <tmpl_string> | default = '{{ template "wechat.default.message" . }}' ]
[ agent_id: <string> | default = '{{ template "wechat.default.agent_id" . }}' ]
[ to_user: <string> | default = '{{ template "wechat.default.to_user" . }}' ]
[ to_party: <string> | default = '{{ template "wechat.default.to_party" . }}' ]
[ to_tag: <string> | default = '{{ template "wechat.default.to_tag" . }}' ]``
複製代碼
Prometheus官網地址:prometheus.io/ 個人Github:github.com/Alrights/pr…