SpringBoot實戰電商項目mall(50k+star)地址:github.com/macrozheng/…linux
以前一直使用的日誌收集方案是ELK,動輒佔用幾個G的內存,有些配置很差的服務器有點頂不住!最近發現一套輕量級日誌收集方案: Loki+Promtail+Grafana(簡稱LPG), 幾百M內存就夠了,並且界面也挺不錯的,推薦給你們!git
LPG日誌收集方案內存佔用不多,經濟且高效!它不像ELK日誌系統那樣爲日誌創建索引,而是爲每一個日誌流設置一組標籤。下面分別介紹下它的核心組件:github
實現這套日誌收集方案須要安裝Loki、Promtail、Grafana這些服務,直接使用
docker-compose
來安裝很是方便。spring
docker-compose.yml
腳本以下,直接使用docker-compose
命令運行便可;version: "3"
services:
# 日誌存儲和解析
loki:
image: grafana/loki
container_name: lpg-loki
volumes:
- /mydata/loki/:/etc/loki/
# 修改loki默認配置文件路徑
command: -config.file=/etc/loki/loki.yml
ports:
- 3100:3100
# 日誌收集器
promtail:
image: grafana/promtail
container_name: lpg-promtail
volumes:
# 將須要收集的日誌所在目錄掛載到promtail容器中
- /mydata/app/mall-tiny-loki/logs/:/var/log/
- /mydata/promtail:/etc/promtail/
# 修改promtail默認配置文件路徑
command: -config.file=/etc/promtail/promtail.yml
# 日誌可視化
grafana:
image: grafana/grafana
container_name: lpg-grafana
ports:
- 3000:3000
複製代碼
因爲咱們把Loki和Promtail的配置文件掛載到了宿主機上,在運行以前,須要先準備好這兩個配置文件;docker
Loki的配置文件/mydata/loki/loki.yml
內容以下,使用的是默認配置(能夠先不掛載配置文件運行Loki的Docker容器,而後從容器中拷貝出來便可);api
auth_enabled: false
server:
http_listen_port: 3100
ingester:
lifecycler:
address: 127.0.0.1
ring:
kvstore:
store: inmemory
replication_factor: 1
final_sleep: 0s
chunk_idle_period: 1h # Any chunk not receiving new logs in this time will be flushed
max_chunk_age: 1h # All chunks will be flushed when they hit this age, default is 1h
chunk_target_size: 1048576 # Loki will attempt to build chunks up to 1.5MB, flushing first if chunk_idle_period or max_chunk_age is reached first
chunk_retain_period: 30s # Must be greater than index read cache TTL if using an index cache (Default index read cache TTL is 5m)
max_transfer_retries: 0 # Chunk transfers disabled
schema_config:
configs:
- from: 2020-10-24
store: boltdb-shipper
object_store: filesystem
schema: v11
index:
prefix: index_
period: 24h
storage_config:
boltdb_shipper:
active_index_directory: /loki/boltdb-shipper-active
cache_location: /loki/boltdb-shipper-cache
cache_ttl: 24h # Can be increased for faster performance over longer query periods, uses more disk space
shared_store: filesystem
filesystem:
directory: /loki/chunks
compactor:
working_directory: /loki/boltdb-shipper-compactor
shared_store: filesystem
limits_config:
reject_old_samples: true
reject_old_samples_max_age: 168h
chunk_store_config:
max_look_back_period: 0s
table_manager:
retention_deletes_enabled: false
retention_period: 0s
ruler:
storage:
type: local
local:
directory: /loki/rules
rule_path: /loki/rules-temp
alertmanager_url: http://localhost:9093
ring:
kvstore:
store: inmemory
enable_api: true
複製代碼
/mydata/loki/promtail.yml
內容以下,使用的也是默認配置,這裏的clients.url
須要注意下,因爲咱們使用的是docker-compose
部署,因此能夠將服務名稱loki
做爲域名來訪問Loki服務;server:
http_listen_port: 9080
grpc_listen_port: 0
positions:
filename: /tmp/positions.yaml
clients:
- url: http://loki:3100/loki/api/v1/push
scrape_configs:
- job_name: system
static_configs:
- targets:
- localhost
labels:
job: varlogs
__path__: /var/log/*log
複製代碼
docker-compose.yml
腳本安裝全部服務,使用以下命令便可;docker-compose up -d
複製代碼
docker ps |grep lpg
命令查看服務狀態。[root@local-linux lpg]# docker ps |grep lpg
64761b407423 grafana/loki "/usr/bin/loki -conf…" 3 minutes ago Up 3 minutes 0.0.0.0:3100->3100/tcp lpg-loki
67f0f0912971 grafana/grafana "/run.sh" 3 minutes ago Up 3 minutes 0.0.0.0:3000->3000/tcp lpg-grafana
f2d78eb188d1 grafana/promtail "/usr/bin/promtail -…" 3 minutes ago Up 3 minutes lpg-promtail
複製代碼
接下來咱們將使用LPG日誌收集系統來收集SpringBoot應用的日誌,SpringBoot應用基本不用作特殊配置。bash
application.yml
,將日誌輸出到/var/logs
目錄下;spring:
application:
name: mall-tiny-loki
logging:
path: /var/logs
level:
com.macro.mall.tiny: debug
複製代碼
docker run -p 8088:8088 --name mall-tiny-loki \
-v /etc/localtime:/etc/localtime \
-v /mydata/app/mall-tiny-loki/logs:/var/logs \
-e TZ="Asia/Shanghai" \
-d mall-tiny/mall-tiny-loki:1.0-SNAPSHOT
複製代碼
admin:admin
,登陸成功後須要添加Loki爲數據源,訪問地址:http://192.168.7.149:3000/Save&test
保存並測試,顯示綠色提示信息表示設置成功,Loki訪問地址:http://192.168.7.149:3100Explore
選擇Loki,並輸入查詢表達式(Loki query)爲{filename="/var/log/spring.log"}
,就能夠查看咱們的SpringBoot應用輸出的日誌了。本文主要介紹了LPG日誌系統的搭建及使用它收集SpringBoot應用的日誌,LPG日誌收集方案確實很是輕量級,性能也不錯!不過若是你有對日誌進行全文搜索的需求的話,仍是得使用ELK系統。若是你對Grafana還不熟悉的話,能夠參考下這篇文章《號稱下一代可視化監控系統,結合SpringBoot使用,賊爽!》。服務器
github.com/macrozheng/…markdown
本文 GitHub github.com/macrozheng/… 已經收錄,歡迎你們Star!app