1. Intro

最近在瞭解日誌聚合系統，正好前幾天看到一篇文章《用了日誌系統新貴Loki，ELK忽然不香了！》，因此就決定動手體驗一下。本文就帶你們快速瞭解下Loki，並簡單介紹.NET Core如何集成Loki。node

2. What's Loki Stack

Grafana Loki like Prometheus, but for logs。其是一個水平可擴展，高可用性，多租戶的日誌聚合系統，基於Apatch 2.0開源。其有三部分組成：git

Loki 是主服務器，負責存儲日誌和處理查詢。對標ELK中的ElasticSearch。
Promtail 是代理，負責收集日誌並將其發送給loki。對標ELK中的Logstash。
Grafana提供用戶界面。對標ELK中的Kibana。

3. Why Use Loki

日誌聚合系統的目的是爲了方便咱們進行日誌跟蹤和故障排查，尤爲在雲原生的環境之下。目前主流的日誌聚合系統，當數ELK、EFK和Loki。Loki相較於ELK Stack有如下優點：github

Elasticsearch中的數據做爲非結構化JSON對象存儲在磁盤上，Loki以二進制的形式存儲。web
Elasticsearch採用全文索引，倒排索引的切分和共享的成本較高。Loki僅索引元數據，好比標籤。json
和Prometheus無縫集成。windows

4. How Use Loki

首先咱們先來基於Heml安裝Loki到本地K8S集羣。api

1. 添加Loki Chart 倉庫：瀏覽器

PS C:\Users\Shengjie> helm repo add loki https://grafana.github.io/loki/charts"loki" has been added to your repositoriesPS C:\Users\Shengjie> helm repo updateHang tight while we grab the latest from your chart repositories......Successfully got an update from the "stable" chart repository...Successfully got an update from the "loki" chart repositoryUpdate Complete. ⎈ Happy Helming!⎈

2. 安裝Loki Stackruby

PS C:\Users\Shengjie> helm search hub loki-stackURL CHART VERSION APP VERSION DESCRIPTIONhttps://hub.helm.sh/charts/loki/loki-stack 0.38.3 v1.5.0 Loki: like Prometheus, but for logs.PS C:\Users\Shengjie> helm show values loki/loki-stackloki: enabled: true
promtail: enabled: true
fluent-bit: enabled: false
grafana: enabled: false sidecar: datasources: enabled: true image: tag: 6.7.0
prometheus: enabled: false
filebeat: enabled: false filebeatConfig: filebeat.yml: | # logging.level: debug filebeat.inputs: - type: container paths: - /var/log/containers/*.log processors: - add_kubernetes_metadata: host: ${NODE_NAME} matchers: - logs_path: logs_path: "/var/log/containers/" output.logstash: hosts: ["logstash-loki:5044"]
logstash: enabled: false image: repository: grafana/logstash-output-loki tag: 1.0.1 filters: main: |- filter { if [kubernetes] { mutate { add_field => { "container_name" => "%{[kubernetes][container][name]}" "namespace" => "%{[kubernetes][namespace]}" "pod" => "%{[kubernetes][pod][name]}" } replace => { "host" => "%{[kubernetes][node][name]}"} } } mutate { remove_field => ["tags"] } } outputs: main: |- output { loki { url => "http://loki:3100/loki/api/v1/push" #username => "test" #password => "test" } # stdout { codec => rubydebug } }

從上面的Values中，能夠看出，能夠自定義啓用fluent-bit、grafana、filebeat、prometheus、logstash組件。這裏咱們僅啓用grafana日誌界面。服務器

PS C:\Users\Shengjie> helm install loki-stack loki/loki-stack --set grafana.enabled=truecoalesce.go:165: warning: skipped value for filters: Not a table.coalesce.go:165: warning: skipped value for filters: Not a table.NAME: loki-stackLAST DEPLOYED: Sun Jul 26 11:58:11 2020NAMESPACE: defaultSTATUS: deployedREVISION: 1NOTES:The Loki stack has been deployed to your cluster. Loki can now be added as a datasource in Grafana.
See http://docs.grafana.org/features/datasources/loki/ for more detail.

3. 登陸Grafana

首先確認Loki是否成功部署：

PS C:\Users\Shengjie> helm listNAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSIONloki-stack default 1 2020-07-26 11:58:11.022896 +0800 CST deployed loki-stack-0.38.3 v1.5.0PS C:\Users\Shengjie> kubectl get pod -wNAME READY STATUS RESTARTS AGEloki-stack-0 1/1 Running 0 2m33sloki-stack-grafana-c447cfbd-z6tbg 1/1 Running 0 2m33sloki-stack-promtail-j47hl 1/1 Running 0 2m33sPS C:\Users\Shengjie> kubectl get svc -wNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEkubernetes ClusterIP 10.96.0.1 <none> 443/TCP 56dloki-stack ClusterIP 10.110.83.209 <none> 3100/TCP 2m52sloki-stack-grafana ClusterIP 10.111.24.26 <none> 80/TCP 2m52sloki-stack-headless ClusterIP None <none> 3100/TCP 2m52s

從上面可知，已經成功啓動，其中loki暴露的容器端口爲3100，grafana暴露的端口爲80。所以咱們下一步須要進行端口轉發，才能訪問grafana。

PS C:\Users\Shengjie> kubectl port-forward svc/loki-stack-grafana 3000:80Forwarding from 127.0.0.1:3000 -> 3000Forwarding from [::1]:3000 -> 3000

而後本地瀏覽器打開http://localhost:3000/就能夠訪問了。其默認用戶是admin，默認密碼在哪裏呢，別慌，咱們去看下loki-stack chart 的readme：

PS C:\Users\Shengjie> helm show readme loki/loki-stack# Loki-Stack Helm Chart## Prerequisites
Make sure you have Helm [installed](https://helm.sh/docs/using_helm/#installing-helm) and[deployed](https://helm.sh/docs/using_helm/#installing-tiller) to your cluster. Then addLoki's chart repository to Helm:$ helm repo add loki https://grafana.github.io/loki/charts
You can update the chart repository by running:$ helm repo update
## Deploy Loki and Promtail to your cluster### Deploy with default config$ helm upgrade --install loki loki/loki-stack### Deploy in a custom namespace$ helm upgrade --install loki --namespace=loki-stack loki/loki-stack
### Deploy with custom config
$ helm upgrade --install loki loki/loki-stack --set "key1=val1,key2=val2,..."## Deploy Loki and Fluent Bit to your cluster$ helm upgrade --install loki loki/loki-stack \ --set fluent-bit.enabled=true,promtail.enabled=false## Deploy Grafana to your clusterThe chart loki-stack contains a pre-configured Grafana, simply use `--set grafana.enabled=true`
To get the admin password for the Grafana pod, run the following command:$ kubectl get secret --namespace <YOUR-NAMESPACE> loki-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echoTo access the Grafana UI, run the following command:$ kubectl port-forward --namespace <YOUR-NAMESPACE> service/loki-grafana 3000:80Navigate to http://localhost:3000 and login with `admin` and the password output above.Then follow the [instructions for adding the loki datasource](/docs/getting-started/grafana.md), using the URL `http://loki:3100/`

上面已經說了很清楚了，能夠從secret中獲取。

若是在windows powersheel中執行，須要分兩步：1. 先獲取base64加密的密碼PS C:\Users\Shengjie> $pwd= kubectl get secret --namespace default loki-stack-grafana -o jsonpath="{.data.admin-password}"2. decode base64PS C:\Users\Shengjie> [Text.Encoding]::Utf8.GetString([Convert]::FromBase64String($pwd))CjnbkkQmwQynZ96gCsynSf0elYQLOp4dyuDnp9jJ
------------------------Linux命令行執行：shengjie@Thinkpad:/mnt/c/Users/Shengjie$ kubectl get secret --namespace default loki-stack-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echoCjnbkkQmwQynZ96gCsynSf0elYQLOp4dyuDnp9jJ

而後使用admin/CjnbkkQmwQynZ96gCsynSf0elYQLOp4dyuDnp9jJ便可成功登陸http://localhost:3000/。

5. Use Loki With .NET Core

下一步，咱們就來建立一個ASP.NET Core Web 應用，將日誌記錄到Loki，並經過Grafana進行聚合分析。

PS C:\Users\Shengjie> dotnet new web -n Loki.K8s.DemoThe template "ASP.NET Core Empty" was created successfully.
Processing post-creation actions...Running 'dotnet restore' on Loki.K8s.Demo\Loki.K8s.Demo.csproj... Determining projects to restore... Restored C:\Users\Shengjie\Loki.K8s.Demo\Loki.K8s.Demo.csproj (in 150 ms).
Restore succeeded.PS C:\Users\Shengjie> cd .\Loki.K8s.Demo\# 添加Serilog.AspNetCore和Serilog.Sinks.Loki Nuget包。PS C:\Users\Shengjie\Loki.K8s.Demo> dotnet add package Serilog.AspNetCorePS C:\Users\Shengjie\Loki.K8s.Demo> dotnet add package Serilog.Sinks.Loki

從上可知，日誌組件選用的是Serilog，由於其支持持久化日誌到Loki。修改Program.cs以下：

public static IHostBuilder CreateHostBuilder(string[] args) => Host.CreateDefaultBuilder(args) .ConfigureWebHostDefaults(webBuilder => { webBuilder.UseStartup<Startup>(); }).UseSerilog((ctx, cfg) => { //cfg.MinimumLevel.Override("Microsoft", LogEventLevel.Warning);//Microsoft框架自己的日誌，僅輸出Warning以上級別 cfg.Enrich.FromLogContext() .Enrich.WithProperty("App", ctx.HostingEnvironment.ApplicationName) .Enrich.WithProperty("ENV", ctx.HostingEnvironment.EnvironmentName) .WriteTo.LokiHttp(new NoAuthCredentials("http://localhost:3100"))//配置Loki Url和認證方式 .WriteTo.Console(); });

修改Startup.cs的Configure方法以下：

public void Configure(IApplicationBuilder app, IWebHostEnvironment env,ILogger<Startup> logger){ if (env.IsDevelopment()) { app.UseDeveloperExceptionPage(); } app.UseRouting(); app.UseEndpoints(endpoints => { endpoints.MapGet("/", async context => { logger.LogInformation("start handle request!"); await context.Response.WriteAsync("Hello World!"); logger.LogInformation("end handle request!"); }); });}

由於不想把應用打包成鏡像運行到K8S中，因此咱們須要把K8S的Loki服務作一次端口轉發暴露到本機，轉發後，就能夠使用http://localhost:3100做爲Loki的Url進行日誌寫入啦。

PS C:\Users\Shengjie\Loki.K8s.Demo> kubectl port-forward svc/loki-stack 3100:3100Forwarding from 127.0.0.1:3100 -> 3100Forwarding from [::1]:3100 -> 3100

運行項目後，從新打開Grafana，添加過濾條件，就能夠查看應用日誌了。

參考資料：

日誌聚合工具loki

CSDN-linkt1234-Loki

Loki官方文檔

本文分享自微信公衆號 - 微服務知多少（dotnet-microservice）。
若有侵權，請聯繫 support@oschina.cn 刪除。
本文參與「OSC源創計劃」，歡迎正在閱讀的你也加入，一塊兒分享。

.NET Core + K8S + Loki 玩轉日誌聚合

1. Intro

2. What's Loki Stack

3. Why Use Loki

4. How Use Loki

5. Use Loki With .NET Core