下載jmx_exporter的jar包html
建立配置文件namenode.yaml(datanode.yaml)放在任意位置，內容爲你想要的metricsnode
參考配置：linux
--- startDelaySeconds: 0 hostPort: master:1234 #master爲本機IP（通常可設置爲localhost）；1234爲想設置的jmx端口（可設置爲未被佔用的端口） #jmxUrl: service:jmx:rmi:///jndi/rmi://127.0.0.1:1234/jmxrmi ssl: false lowercaseOutputName: false lowercaseOutputLabelNames: false
其餘參數參考：json
| Name | Description |
|---|---|
| startDelaySeconds | start delay before serving requests. Any requests within the delay period will result in an empty metrics set. |
| hostPort | The host and port to connect to via remote JMX. If neither this nor jmxUrl is specified, will talk to the local JVM. |
| username | The username to be used in remote JMX password authentication. |
| password | The password to be used in remote JMX password authentication. |
| jmxUrl | A full JMX URL to connect to. Should not be specified if hostPort is. |
| ssl | Whether JMX connection should be done over SSL. To configure certificates you have to set following system properties:-Djavax.net.ssl.keyStore=/home/user/.keystore-Djavax.net.ssl.keyStorePassword=changeit-Djavax.net.ssl.trustStore=/home/user/.truststore-Djavax.net.ssl.trustStorePassword=changeit |
| lowercaseOutputName | Lowercase the output metric name. Applies to default format and name. Defaults to false. |
| lowercaseOutputLabelNames | Lowercase the output metric label names. Applies to default format and labels. Defaults to false. |
| whitelistObjectNames | A list of ObjectNames to query. Defaults to all mBeans. |
| blacklistObjectNames | A list of ObjectNames to not query. Takes precedence over whitelistObjectNames. Defaults to none. |
| rules | A list of rules to apply in order, processing stops at the first matching rule. Attributes that aren't matched aren't collected. If not specified, defaults to collecting everything in the default format. |
| pattern | Regex pattern to match against each bean attribute. The pattern is not anchored. Capture groups can be used in other options. Defaults to matching everything. |
| attrNameSnakeCase | Converts the attribute name to snake case. This is seen in the names matched by the pattern and the default format. For example, anAttrName to an_attr_name. Defaults to false. |
| name | The metric name to set. Capture groups from the pattern can be used. If not specified, the default format will be used. If it evaluates to empty, processing of this attribute stops with no output. |
| value | Value for the metric. Static values and capture groups from the pattern can be used. If not specified the scraped mBean value will be used. |
| valueFactor | Optional number that value (or the scraped mBean value if value is not specified) is multiplied by, mainly used to convert mBean values from milliseconds to seconds. |
| labels | A map of label name to label value pairs. Capture groups from pattern can be used in each. name must be set to use this. Empty names and values are ignored. If not specified and the default format is not being used, no labels are set. |
| help | Help text for the metric. Capture groups from pattern can be used. name must be set to use this. Defaults to the mBean attribute decription and the full name of the attribute. |
| type | The type of the metric, can be GAUGE, COUNTER or UNTYPED. name must be set to use this. Defaults to UNTYPED. |
修改$HADOOP_HOME/etc/hadoop/hadoop-env.sh api
NameNode節點添加：oracle
export HADOOP_NAMENODE_OPTS="-Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.port=1234 $HADOOP_NAMENODE_OPTS "
DataNode節點添加：app
export HADOOP_DATANODE_OPTS="-Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.port=1235 $HADOOP_DATANODE_OPTS "
提示：jvm
端口1234（1235）要與以前設置的jmx端口保持一致maven
修改
$HADOOP_HOME/bin/hdfs
export HADOOP_NAMENODE_OPTS="$HADOOP_NAMENODE_OPTS -javaagent:/home/hadoop/jmx_prometheus_javaagent-0.3.1.jar=9200:/home/hadoop/namenode.yaml"
export HADOOP_DATANODE_OPTS="$HADOOP_DATANODE_OPTS -javaagent:/home/hadoop/jmx_prometheus_javaagent-0.3.1.jar=9300:/home/hadoop/datanode.yaml"
提示：9200（9300）爲jmx_exporter提供metrics數據端口，後續Prometheus今後端口獲取數據
訪問http://master:9200/metrics就能得到須要的metrics數據:
# HELP jvm_buffer_pool_used_bytes Used bytes of a given JVM buffer pool. # TYPE jvm_buffer_pool_used_bytes gauge jvm_buffer_pool_used_bytes{pool="direct",} 1181032.0 jvm_buffer_pool_used_bytes{pool="mapped",} 0.0 # HELP jvm_buffer_pool_capacity_bytes Bytes capacity of a given JVM buffer pool. # TYPE jvm_buffer_pool_capacity_bytes gauge jvm_buffer_pool_capacity_bytes{pool="direct",} 1181032.0 jvm_buffer_pool_capacity_bytes{pool="mapped",} 0.0 # HELP jvm_buffer_pool_used_buffers Used buffers of a given JVM buffer pool.
...
下載Prometheus
https://prometheus.io/download/
解壓
修改配置文件 prometheus.yml
添加
- job_name: hadoop-master static_configs: - targets: ['localhost:9200'] - job_name: hadoop-slave1 static_configs: - targets: ['slave1:9300']
運行
./prometheus
http://master:9090/targets查看是否添加成功
成功則增長master，slave1
以下圖
經過點擊http://localhost:9200/metrics能夠看到metrics數據
下載Grafana
https://grafana.com/grafana/download?platform=linux
安裝相應版本grafana
啓動grafana
systemctl start grafana-server
啓動後，便可經過http://master:3000/來訪問了（默認帳號密碼是admin/admin）
以下圖
關聯Grafana和Prometheus
點擊Data Sources
點擊Add data source，填寫數據保存
添加一個dashboard，以下圖進入dashboard導入頁面
點擊Upload .json File，導入模版（可從https://grafana.com/dashboards下載）
導入成功後能夠看到相似以下效果，完成！