docker+efk+.net core部署

部署環境

centos7

本主要利用efk實現日誌收集

1、建立docker-compose

es地址：https://www.elastic.co/guide/en/elasticsearch/reference/6.4/docker.html

fluentd地址：https://hub.docker.com/r/fluent/fluentd

kibana地址：https://www.elastic.co/guide/en/kibana/current/docker.html

一、利用xshell+xftp在centos7的/root/test下建立文件夾掛載容器內配置、數據等

fluentd -config fluent.conf
　　-plugins #空文件夾 　　Dockerfile 　　docker-compose.yml 　　fluent.conf #與上面同樣

二、建立本身的fluentd鏡像 （#由於鏡像中不支持es插件輸出，如下能夠參考上面fluentd地址）

上面目錄中的Dockerfile文件以下：

Dockerfile

FROM fluent/fluentd:v1.3-onbuild-1 # Use root account to use apk USER root # below RUN includes plugin as examples elasticsearch is not required # you may customize including plugins as you wish RUN apk add --no-cache --update --virtual .build-deps \ sudo build-base ruby-dev \ && sudo gem install \ fluent-plugin-elasticsearch \ && sudo gem sources --clear-all \ && apk del .build-deps \ && rm -rf /home/fluent/.gem/ruby/2.5.0/cache/*.gem USER fluent

fluent.conf 能夠根據本身狀況設置默認，由於啓動fluentd的時候會本身加載/fluentd/etc/fluent.conf這個文件。你能夠把它掛在在外面

fluent.conf

<source> @type forward port 24224 bind 0.0.0.0
</source>

<filter> @type parser format json emit_invalid_record_to_error false time_format %Y-%m-%dT%H:%M:%S.%L%Z key_name log </filter>

<match **> @type elasticsearch host 192.168.1.157 port 9200 logstash_format true
</match>

 

cd到 /root/test/fluentd 執行 

生成支持es的fluentd鏡像完畢docker build -t custom-fluentd:latest ./

三、利用docker-compose.yml啓動

version: '3.4' services: elasticsearch: image: docker.elastic.co/elasticsearch/elasticsearch:6.4.3 container_name: elasticsearch environment: discovery.type: "single-node" http.cors.enabled: "true" volumes: - esdata1:/usr/share/elasticsearch/data ports: - "9200:9200"
      - "9300:9300" kibana: image: docker.elastic.co/kibana/kibana:6.4.3 container_name: kibana environment: SERVER_NAME: kibana ELASTICSEARCH_HOSTS: http://192.168.1.157:9200 # default is http://elasticsearch:9200
 ports: - "5601:5601" depends_on: - elasticsearch fluentd: image: custom-fluentd #build: # context: ./fluentd/ # dockerfile: Dockerfile container_name: fluentd ports: - "24224:24224"
      - "24224:24224/udp" volumes: - /root/test/fluentd/log:/fluentd/log - /root/test/fluentd/config:/fluentd/etc depends_on: - elasticsearch volumes: esdata1: driver: local

輸入http://ip:5601查看kibana

輸入http://ip:9200查看es

注意：啓動過程可能會由於es還沒啓動好fluentd就啓動致使fluentd沒連上es能夠經過查看日誌docker logs fluentd肯定是否連上，若是沒連上，能夠經過wait-for-it.sh或wait-for進行延遲編排，本文不講

參考地址：https://my.oschina.net/eacdy/blog/1824219

若是仍是不行能夠把上面的分開一個一個啓動

docker-compose -d up

2、.net core 利用serilog日誌組件輸出到es

一、項目中NuGet 

Serilog.AspNetCore

Serilog.Settings.Configuration

Serilog.Sinks.Console

Serilog.Sinks.Elasticsearch

二、Appsetting.json中配置

{ "Serilog": { "Using":  ["Serilog.Sinks.Console"], "MinimumLevel": "Warning", "WriteTo": [ { "Name": "Console" } ], "Enrich": ["FromLogContext", "WithMachineName", "WithThreadId"], "Destructure": [ { "Name": "ToMaximumDepth", "Args": { "maximumDestructuringDepth": 4 } }, { "Name": "ToMaximumStringLength", "Args": { "maximumStringLength": 100 } }, { "Name": "ToMaximumCollectionCount", "Args": { "maximumCollectionCount": 10 } } ], "Properties": { "Application": "ApplicationName" } } }

 

三、program.cs中配置

....

using Serilog;
using Serilog.Sinks.Elasticsearch;

　　　　public static IWebHostBuilder CreateWebHostBuilder(string[] args) => WebHost.CreateDefaultBuilder(args) .UseKestrel() .UseContentRoot(Directory.GetCurrentDirectory()) .UseIISIntegration()
                .UseStartup<Startup>() .UseSerilog((ctx, config)=> { config.ReadFrom.Configuration(ctx.Configuration); #if DEBUG config.WriteTo.Console(); #else config.WriteTo.Console(new ElasticsearchJsonFormatter()); #endif });

配置好後能夠運行起來，這個時候的控制檯輸出的日誌就已是es類型格式了

四、編寫Dockerfile打包項目鏡像

Dockerfile

FROM microsoft/dotnet:2.2-aspnetcore-runtime AS base WORKDIR /app EXPOSE 80 FROM microsoft/dotnet:2.2-sdk AS publish WORKDIR /src COPY . . RUN dotnet publish -c Release -o /app FROM base AS final WORKDIR /app COPY --from=publish /app . ENTRYPOINT ["dotnet", "ApplicationName.dll"]

 

五、利用docker-compose.yml啓動項目

docker-compose.yml

ApplicationName: image: ApplicationName container_name: ApplicationName build: context: ./ApplicationName/ dockerfile: Dockerfile environment: - ASPNETCORE_URLS=http://0.0.0.0:80
 restart: always ports: - "5000:80" logging: driver: "fluentd" options: fluentd-address: "tcp://192.168.1.157:24224"

其中logging要指定日誌輸出類型及日誌輸出到fluentd的地址端口

把docker-compose.yml放在項目根目錄下，cd到項目根目錄運行

docker-compose -d up

就能夠啓動完成

在瀏覽器中輸入ip:port便可查看

注意：這裏的全部docker-compose.yml都是分開的因此啓動後可能會分佈在不一樣的網絡中，能夠建立一個網絡docker network create netname，而後保證他們在同一個網絡裏面這樣就能夠直接用容器名來鏈接而不須要用宿主機的ip