高可用分佈式clickhouse集羣搭建

介紹clickhouse單機和高可用分佈式搭建

    docker單機搭建方法

#使用yandex的服務端鏡像
docker pull yandex/clickhouse-server
#使用yandex的客戶端鏡像
docker pull yandex/clickhouse-client

yandex/clickhosue-server的Dockerfile以下,可參考進行自定義鏡像搭建

FROM ubuntu:16.04

ARG repository="deb http://repo.yandex.ru/clickhouse/deb/stable/ main/"
ARG version=\*

RUN apt-get update && \
    apt-get install -y apt-transport-https && \
    mkdir -p /etc/apt/sources.list.d && \
    echo $repository | tee /etc/apt/sources.list.d/clickhouse.list && \
    apt-get update && \
    apt-get install --allow-unauthenticated -y clickhouse-server-common=$version clickhouse-server-base=$version && \
    rm -rf /var/lib/apt/lists/* /var/cache/debconf && \
    apt-get clean

COPY docker_related_config.xml /etc/clickhouse-server/config.d/
RUN chown -R clickhouse /etc/clickhouse-server/

USER clickhouse
EXPOSE 9000 8123 9009
VOLUME /var/lib/clickhouse

ENV CLICKHOUSE_CONFIG /etc/clickhouse-server/config.xml

ENTRYPOINT exec /usr/bin/clickhouse-server --config=${CLICKHOUSE_CONFIG}

運行clickhosue服務鏡像 yandex/clickhouse-server

$ docker run -d --name some-clickhouse-server --ulimit nofile=262144:262144 yandex/clickhouse-server

運行clickhouse客戶端鏡像 yandex/clickhouse-client

$ docker run -it --rm --link some-clickhouse-server:clickhouse-server yandex/clickhouse-client --host clickhouse-server

這裏推薦一個可視化的鏈接工具DBeaver https://dbeaver.jkiss.org/download/

能夠直接鏈接到clickhosue  用戶名默認爲 default

    高可用集羣搭建

        我這裏準備三臺機器 系統都是ubuntu server 16.04

            192.168.43.225 yun0

            192.168.43.226 yun1

            192.168.43.227 yun2

    在機器上安裝clickhouse數據庫（若是你是虛擬機的話，能夠先裝一臺 ，再複製倆臺）

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv E0C56BD4    # optional
sudo apt-add-repository "deb http://repo.yandex.ru/clickhouse/deb/stable/ main/"
sudo apt-get update
sudo apt-get install clickhouse-server-common clickhouse-client -y

若是提示沒有app-add-repository,你須要安裝下面倆個

apt-get install python-software-properties
apt-get install software-properties-common

裝完後進行clickhouse集羣的配置

配置/etc/clickhouse-server/config.xml

在60行到70行之間，有一個監聽主機（配置遠程鏈接）

通常把下面這三行註釋去掉就好了

<listen_host>::</listen_host>                                            
<listen_host>::1</listen_host>                                           
<listen_host>127.0.0.1</listen_host>

同時在這個xml文件裏面能夠看到這樣一段註釋

<!-- If element has 'incl' attribute, then for it's value will be used corresponding substi
    tution from another file.
155          By default, path to file with substitutions is /etc/metrika.xml. It could be changed i
    n config in 'include_from' element.
156          Values for substitutions are specified in /yandex/name_of_substitution elements in tha
    t file.
157       -->

metrika.xml就是分佈式配置文件

在etc目錄下新建metrika.xml文件

可直接在下面的實例中修改

<yandex>
<clickhouse_remote_servers>
    <perftest_3shards_1replicas>
        <shard>
             <internal_replication>true</internal_replication>
            <replica>
                <host>yun0</host>
                <port>9000</port>
            </replica>
        </shard>
        <shard>
            <replica>
                <internal_replication>true</internal_replication>
                <host>yun1</host>
                <port>9000</port>
            </replica>
        </shard>
        <shard>
            <internal_replication>true</internal_replication>
            <replica>
                <host>yun2</host>
                <port>9000</port>
            </replica>
        </shard>
    </perftest_3shards_1replicas>
</clickhouse_remote_servers>


<zookeeper-servers>
  <node index="1">
    <host>yun0</host>
    <port>2181</port>
  </node>

  <node index="2">
    <host>yun1</host>
    <port>2181</port>
  </node>
  <node index="3">
    <host>yun2</host>
    <port>2181</port>
  </node>
</zookeeper-servers>

<macros>
    <replica>yun0</replica>
</macros>


<networks>
   <ip>::/0</ip>
</networks>


<clickhouse_compression>
<case>
  <min_part_size>10000000000</min_part_size>
                                             
  <min_part_size_ratio>0.01</min_part_size_ratio>                                                                                                                                       
  <method>lz4</method>
</case>
</clickhouse_compression>

</yandex>

這個配置主要修改三個地方 

1.配置節點

<clickhouse_remote_servers>
    <perftest_3shards_1replicas>
        <shard>
             <internal_replication>true</internal_replication>
            <replica>
                <host>yun0</host>
                <port>9000</port>
            </replica>
        </shard>
        <shard>
            <replica>
                <internal_replication>true</internal_replication>
                <host>yun1</host>
                <port>9000</port>
            </replica>
        </shard>
        <shard>
            <internal_replication>true</internal_replication>
            <replica>
                <host>yun2</host>
                <port>9000</port>
            </replica>
        </shard>
    </perftest_3shards_1replicas>
</clickhouse_remote_servers>

2.配置zookeeper 

<zookeeper-servers>
  <node index="1">
    <host>yun0</host>
    <port>2181</port>
  </node>

  <node index="2">
    <host>yun1</host>
    <port>2181</port>
  </node>
  <node index="3">
    <host>yun2</host>
    <port>2181</port>
  </node>
</zookeeper-servers>

3.前面倆個都沒啥好說的，下面這個注意每一個節點填當前節點的ip

<macros>
    <replica>yun0</replica>
</macros>

後面就是安裝jdk,zookeeper.這倆個就不介紹了。

裝完後重啓  啓動zookerper，clickhouse-server服務。