k8s與log--利用lua爲fluent bit添加一個filter

前言

以前咱們介紹過fluent bit這個日誌收集神器。最近咱們遇到奇葩的需求，不得不利用lua編寫fluent bit的filter，來知足需求。

首先介紹一下需求：
非容器的日誌團隊使用filebeat, 其配置文件部分以下：

processors:
- dissect:
    tokenizer: "/data/logs/%{appname}/%{filename}.log"
    field: "source"
    target_prefix: ""

即須要從日誌record的source filed 提取appname和filename兩個filed。

fluent bit 並無如此的插件，因此不得不本身實現。

實現

lua編寫filter規範

官方給出的示例以下：

function cb_print(tag, timestamp, record)
   return code, timestamp, record
end

Function 輸入參數

Function Arguments

name	description
tag	Name of the tag associated with the incoming record.
timestamp	Unix timestamp with nanoseconds associated with the incoming record. The original format is a double (seconds.nanoseconds)
record	Lua table with the record content

Return Values

Each callback must return three values:

name	data type	description
code	integer	The code return value represents the result and further action that may follows. If code equals -1, means that filter_lua must drop the record. If _code_ equals 0 the record will not be modified, otherwise if code equals 1, means the original timestamp or record have been modified so it must be replaced by the returned values from timestamp (second return value) and record (third return value).
timestamp	double	If code equals 1, the original record timestamp will be replaced with this new value.
record	table	if code equals 1, the original record information will be replaced with this new value. Note that the format of this value must be a valid Lua table.

理解上面的規範能夠結合下面的寫法。注意返回值的code。

代碼實現

編寫實現相似功能的lua文件，以下：

function dissect(tag, timestamp, record)
    source = record["source"]
    if (source == nil)
    then   
      return 0, 0, 0
    else 
        new_record = record

        local result = { }   
        local from  = 1
        local delim_from, delim_to = string.find( source, "/", from  )
        while delim_from do
            table.insert( result, string.sub( source, from , delim_from-1 ) )
            from  = delim_to + 1
            delim_from, delim_to = string.find( source, "/", from  )
        end
        table.insert( result, string.sub( source, from  ) )
        new_record["appname"] = result[7]
        new_record["filename"] = string.sub( result[8], 1, -5 )
        return 1, timestamp, new_record
    end
 end

備註：

• 在咱們k8s環境下，業務日誌掛盤路徑相似於下面的格式：source = /data/logs/default/tomcat/742473c7-17dc-11e9-afc5-0a07a5c4fbe2/appname/filename.log
• result[7]之因此出現這種及其容易出現bug的寫法，一是因爲咱們這邊有嚴格的日誌規範，另外，只是給你們提供一種lua寫filter的思路。

製做鏡像

咱們是基於fluent bit 1.0.2 。因此找到官方的代碼倉庫，git clone 下來，稍做更改。
新的dockerfile以下：

FROM debian:stretch as builder

# Fluent Bit version
ENV FLB_MAJOR 1
ENV FLB_MINOR 0
ENV FLB_PATCH 2
ENV FLB_VERSION 1.0.2

ENV DEBIAN_FRONTEND noninteractive

ENV FLB_TARBALL http://github.com/fluent/fluent-bit/archive/v$FLB_VERSION.zip
RUN mkdir -p /fluent-bit/bin /fluent-bit/etc /fluent-bit/log /tmp/fluent-bit-master/

RUN apt-get update && \
    apt-get install -y --no-install-recommends \
      build-essential \
      cmake \
      make \
      wget \
      unzip \
      libssl1.0-dev \
      libasl-dev \
      libsasl2-dev \
      pkg-config \
      libsystemd-dev \
      zlib1g-dev \
      ca-certificates \
    && wget -O "/tmp/fluent-bit-${FLB_VERSION}.zip" ${FLB_TARBALL} \
    && cd /tmp && unzip "fluent-bit-$FLB_VERSION.zip" \
    && cd "fluent-bit-$FLB_VERSION"/build/ \
    && rm -rf /tmp/fluent-bit-$FLB_VERSION/build/*

WORKDIR /tmp/fluent-bit-$FLB_VERSION/build/
RUN cmake -DFLB_DEBUG=On \
          -DFLB_TRACE=Off \
          -DFLB_JEMALLOC=On \
          -DFLB_TLS=On \
          -DFLB_SHARED_LIB=Off \
          -DFLB_EXAMPLES=Off \
          -DFLB_HTTP_SERVER=On \
          -DFLB_IN_SYSTEMD=On \
          -DFLB_OUT_KAFKA=On ..

RUN make -j $(getconf _NPROCESSORS_ONLN)
RUN install bin/fluent-bit /fluent-bit/bin/

# Configuration files
COPY fluent-bit.conf \
     parsers.conf \
     parsers_java.conf \
     parsers_extra.conf \
     parsers_openstack.conf \
     parsers_cinder.conf \
     plugins.conf \
     /fluent-bit/etc/

COPY dissect.lua /fluent-bit/bin/

FROM gcr.io/distroless/cc
MAINTAINER Eduardo Silva <eduardo@treasure-data.com>
LABEL Description="Fluent Bit docker image" Vendor="Fluent Organization" Version="1.1"

COPY --from=builder /usr/lib/x86_64-linux-gnu/*sasl* /usr/lib/x86_64-linux-gnu/
COPY --from=builder /usr/lib/x86_64-linux-gnu/libz* /usr/lib/x86_64-linux-gnu/
COPY --from=builder /lib/x86_64-linux-gnu/libz* /lib/x86_64-linux-gnu/
COPY --from=builder /usr/lib/x86_64-linux-gnu/libssl.so* /usr/lib/x86_64-linux-gnu/
COPY --from=builder /usr/lib/x86_64-linux-gnu/libcrypto.so* /usr/lib/x86_64-linux-gnu/
# These below are all needed for systemd
COPY --from=builder /lib/x86_64-linux-gnu/libsystemd* /lib/x86_64-linux-gnu/
COPY --from=builder /lib/x86_64-linux-gnu/libselinux.so* /lib/x86_64-linux-gnu/
COPY --from=builder /lib/x86_64-linux-gnu/liblzma.so* /lib/x86_64-linux-gnu/
COPY --from=builder /usr/lib/x86_64-linux-gnu/liblz4.so* /usr/lib/x86_64-linux-gnu/
COPY --from=builder /lib/x86_64-linux-gnu/libgcrypt.so* /lib/x86_64-linux-gnu/
COPY --from=builder /lib/x86_64-linux-gnu/libpcre.so* /lib/x86_64-linux-gnu/
COPY --from=builder /lib/x86_64-linux-gnu/libgpg-error.so* /lib/x86_64-linux-gnu/

COPY --from=builder /fluent-bit /fluent-bit

#
EXPOSE 2020

# Entry point
CMD ["/fluent-bit/bin/fluent-bit", "-c", "/fluent-bit/etc/fluent-bit.conf"]

注意增長了 COPY dissect.lua /fluent-bit/bin/ 。

而後就build鏡像便可。

使用姿式

使用比較簡單的。demo以下：

[FILTER]
        Name                lua
        Match               app.*
        script              /fluent-bit/bin/dissect.lua
        call                dissect

• script lua腳本的存放路徑。
• call 即爲lua函數名。

總結

經過寫這個filter，有一下幾個感悟吧。

• 官方的鏡像基於谷歌的distroless鏡像，沒有shell，沒有包管理，調試起來很費力。平時的業務容器化場景中，明顯的不合適，與阿里的富容器思惟南轅北轍。固然除非你公司的業務開發能力足夠強。
• fluent bit 相關的資料仍是有點少。遇到問題和使用一些不明白的地方，解決起來費力。除非你是c專家。官方文檔寫的也不夠詳細，只是描述了個大概。