greenplum集群環(huán)境部署

一同欠、環(huán)境準備

1、準備10臺機器

gp-master
gp-standby
gp-s1
gp-s2
gp-s3
gp-s4
gp-s5
gp-s6
gp-s7
gp-s8
安裝centos6.8

2价卤、更改服務(wù)器hosts

#vim /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.66.33 gp-master
192.168.66.34 gp-standby
192.168.66.35 gp-s1
192.168.66.36 gp-s2
192.168.66.37 gp-s3
192.168.66.38 gp-s4
192.168.66.39 gp-s5
192.168.66.40 gp-s6
192.168.66.41 gp-s7
192.168.66.42 gp-s8

3、更改服務(wù)器之間訪問不適用密碼

在每臺服務(wù)器上都需要執(zhí)行

#ssh-keygen -t rsa #使用root用戶運行给猾，四個回車OK
#ssh-copy-id gp-standby #會提示輸入密碼,ip1是需要ssh免密碼登錄的機器IP
#ssh gp-standby #驗證是否成功

4住册、每臺服務(wù)器添加分組及用戶信息

#groupadd -g 530 gpadmin
#useradd -g 530 -u 530 -m -d /home/gpadmin -s /bin/bash gpadmin
#chown -R gpadmin.gpadmin /home/gpadmin
#echo "123456" | passwd --stdin gpadmin

5咕晋、修改系統(tǒng)內(nèi)核

#vim /etc/sysctl.conf
 
kernel.shmmax=34359738368
kernel.shmall=8388608
kernel.shmmni = 4096
kernel.sem = 250 512000 100 2048
kernel.sysrq = 1
kernel.core_uses_pid = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.msgmni = 2048
net.ipv4.tcp_syncookies = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.conf.all.arp_filter = 1
net.ipv4.ip_local_port_range = 10000 65535
net.core.netdev_max_backlog = 10000
net.core.rmem_max = 2097152
net.core.wmem_max = 2097152
vm.overcommit_memory = 2
vm.overcommit_ratio=95

讓配置生效：# sysctl -p（讓配置生效）

6雹拄、配置進程文件資源限制

#vim /etc/security/limit.conf
 
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072

#vim /etc/security/limits.d/90-nproc.conf 
* soft nproc 131072
* hard nproc 131072
* soft nofile 1048576
* hard nofile 1048576

磁盤預(yù)讀參數(shù)及 deadline算法修改

blockdev --setra 65536 /dev/sda
echo deadline > /sys/block/sda/queue/scheduler

7、防火墻及SELinux配置管理

CentOS 6.8：
關(guān)閉防火墻： service iptables stop
關(guān)閉開機啟動防火墻：chkconfig iptables off
查看防火墻狀態(tài)： service iptables status

SELinux配置：

#vi /etc/selinux/config 
 
 
 
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
#     targeted - Targeted processes are protected,
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

二掌呜、部署安裝Greenplum DB

1滓玖、下載安裝包

下載地址：https://github.com/greenplum-db/gpdb/releases

2、下載 greenplum-db-6.7.0-rhel6-x86_64.rpm

安裝依賴包

sudo yum install -y apr apr-util bash bzip2 curl krb5 libcurl libevent libxml2 libyaml zlib openldap openssh openssl openssl-libs perl readline rsync R sed tar zip krb5-devel

3质蕉、將greenplum的安裝路徑給gpadmin訪問權(quán)限

# chown -R gpadmin.gpadmin /usr/local/
# chown -R gpadmin.gpadmin /usr/local/greenplum-db
# chown -R gpadmin.gpadmin /opt/

三势篡、配置Greenplum DB

1.1創(chuàng)建一個hostlist,包含所有節(jié)點主機名

#vi /home/gpadmin/conf/hostlist
gp-master
gp-standby 
gp-s1 
gp-s2
gp-s3
gp-s4
gp-s5
gp-s6
gp-s7 
gp-s8

1.2創(chuàng)建一個seg_hosts,包含所有的Segment Host的主機名

# vim /home/gpadmin/conf/seg_hosts
gp-s1 
gp-s2
gp-s3
gp-s4
gp-s5
gp-s6
gp-s7 
gp-s8

2、配置ssh免密連接:

[root@ gp-master ~]# su - gpadmin
[gpadmin@ gp-master ~]# source /usr/local/greenplum-db/greenplum_path.sh  
[gpadmin@ gp-master ~]# gpssh-exkeys -f /home/gpadmin/conf/hostlist

3模暗、在segment節(jié)點上安裝Greenplum DB

在各個子節(jié)點進行文件夾賦權(quán)：

$ chown -R gpadmin:gpadmin /usr/local
$ chown -R gpadmin:gpadmin /opt

在主節(jié)點打包安裝包并復(fù)制到各個子節(jié)點：

$ cd /usr/local/
$ tar -cf gp.tar greenplum-db-5.0.0/
$ gpscp -f /home/gpadmin/conf/seg_hosts gp.tar =:/usr/local/

如果沒有意外禁悠，就批量復(fù)制成功了，可以去子節(jié)點的相應(yīng)文件夾查看兑宇，之后要將tar包解壓碍侦，現(xiàn)在我們將采用對子節(jié)點使用批量解壓操作：

$ source /usr/local/ greenplum-db/greenplum_path.sh
$ gpssh -f /home/gpadmin/conf/seg_hosts  
 
=> cd /usr/local
[gp-s1]
[gp-s2]
[gp-s3]
[gp-s4]
[gp-s5]
[gp-s6]
[gp-s7]
[gp-s8]
=> tar -xf gp.tar
[gp-s1]
[gp-s2]
[gp-s3]
[gp-s4]
[gp-s5]
[gp-s6]
[gp-s7]
[gp-s8]
 
#建立軟鏈接
=> ln -s ./greenplum-db-6.7.0 greenplum-db
[gp-s1]
[gp-s2]
[gp-s3]
[gp-s4]
[gp-s5]
[gp-s6]
[gp-s7]
[gp-s8]
=> ll(可以使用ll查看一下是否已經(jīng)安裝成功)
=>exit(退出)

四、初始化數(shù)據(jù)庫

1隶糕、創(chuàng)建資源目錄

source /usr/local/ greenplum-db/greenplum_path.sh
gpssh -f /home/gpadmin/conf/hostlist #統(tǒng)一處理所有節(jié)點
 
#創(chuàng)建資源目錄 /opt/greenplum/data下一系列目錄（生產(chǎn)目錄個數(shù)可根據(jù)需求生成）
=> mkdir -p /opt/greenplum/data/master
=> mkdir -p /opt/greenplum/data/primary
=> mkdir -p /opt/greenplum/data/mirror
=> mkdir -p /opt/greenplum/data2/primary
=> mkdir -p /opt/greenplum/data2/mirror

2瓷产、環(huán)境變量配置

2.1在主節(jié)點進行環(huán)境變量配置（配置在gpadmin的用戶權(quán)限下進行配置）

1). 先切換gpadmin用戶
sudo su gpadmin

2).再添加內(nèi)容到~/.bashrc
cat >> ~/.bashrc << EOF
source /usr/local/greenplum-db/greenplum_path.sh
export MASTER_DATA_DIRECTORY=/opt/greenplum/data/master/gpseg-1
export GPPORT=5432
export PGDATABASE=gp_sydb
EOF

2.2然后依次復(fù)制到各個子節(jié)點

# scp /home/gpadmin/.bash_profile gp-sdw1:/home/gpadmin/

2.3 讓環(huán)境變量生效

# source .bash_profile

3 、NTP 配置

啟用master節(jié)點上的ntp枚驻，并在Segment節(jié)點上配置和啟用NTP：

echo "server master perfer" >>/etc/ntp.conf
gpssh -f /home/gpadmin/conf/hostlist -v -e 'sudo ntpd'
gpssh -f /home/gpadmin/conf/hostlist -v -e 'sudo /etc/init.d/ntpd start && sudo chkconfig --level 35 ntpd on'

4濒旦、初始化前檢查連通性

cd /usr/local/greenplum-db/bin
gpcheckperf -f /home/gpadmin/conf/hostlist -r N -d /tmp

檢查節(jié)點與節(jié)點之間文件讀取再登；

image.png

5尔邓、執(zhí)行初始化

初始化 Greenplum 配置文件模板都在/usr/local/greenplum-db/docs/cli_help/gpconfigs目錄下,gpinitsystem_config是初始化 Greenplum 的模板，此模板中 Mirror Segment的配置都被注釋锉矢；創(chuàng)建一個副本梯嗽，對其以下配置進行修改：

# cd /usr/local/greenplum-db/docs/cli_help/gpconfigs
# cp gpinitsystem_config initgp_config
# vi initgp_config

以下為文本要修改的屬性字段配置
資源目錄為在前面章節(jié)創(chuàng)建的資源目錄，配置幾次資源目錄就是每個子節(jié)點有幾個實例

declare -a DATA_DIRECTORY=(/opt/greenplum/data/primary /opt/greenplum/data/primary /opt/greenplum/data/primary /opt/greenplum/data2/primary /opt/greenplum/data2/primary /opt/greenplum/data2/primary)
declare -a MIRROR_DATA_DIRECTORY=(/opt/greenplum/data/mirror /opt/greenplum/data/mirror /opt/greenplum/data/mirror /opt/greenplum/data2/mirror /opt/greenplum/data2/mirror /opt/greenplum/data2/mirror)
 
ARRAY_NAME=”gp_sydb”                                       
MASTER_HOSTNAME=gp-master                                  
MASTER_DIRECTORY=/opt/greenplum/data/master                 
MASTER_DATA_DIRECTORY=/opt/greenplum/data/master/gpseg-1    
DATABASE_NAME=gp_sydb  
MACHINE_LIST_FILE=/home/gpadmin/conf/seg_hosts

6沽损、執(zhí)行初始化慷荔；

$ gpinitsystem -c initgp_config -s gp-standby

若初始化失敗，需要刪除/opt下的數(shù)據(jù)資源目錄重新初始化缠俺；

五显晶、數(shù)據(jù)庫操作

1 停止和啟動集群

# gpstop -M fast
# gpstart -a

2 登錄數(shù)據(jù)庫

# psql -d postgres  #進入某個數(shù)據(jù)庫
 
postgres=# \l # 查詢數(shù)據(jù)庫
                 List of databases
   Name    |  Owner  | Encoding |  Access privileges  
-----------+---------+----------+---------------------
 gp_sydb   | gpadmin | UTF8     | 
 postgres  | gpadmin | UTF8     | 
 template0 | gpadmin | UTF8     | =c/gpadmin          
                                : gpadmin=CTc/gpadmin
 template1 | gpadmin | UTF8     | =c/gpadmin          
                                : gpadmin=CTc/gpadmin
(4 rows)
postgres=# \i test.sql #執(zhí)行sql
postgres=# copy 表名 to '/tmp/1.csv' with 'csv';      #快速導出單表數(shù)據(jù)
postgres=# copy 表名 from '/tmp/1.csv' with 'csv';    #快速導入單表數(shù)據(jù)
postgres=# \q          #退出數(shù)據(jù)庫

image.png

3 集群狀態(tài)

gpstate -e #查看mirror的狀態(tài)
gpstate -f #查看standby master的狀態(tài)
gpstate -s #查看整個GP群集的狀態(tài)
gpstate -i #查看GP的版本
gpstate --help #幫助文檔，可以查看gpstate更多用法

備注：目前為止數(shù)據(jù)庫已經(jīng)操作完畢壹士，默認只有本地可以連數(shù)據(jù)庫磷雇。如果需要別的I可以連，需要修改gp_hba.conf文件

vim /opt/greenplum/data/master/gpseg-1/pg_hba.conf
#添加
host all gpadmin 0.0.0.0/0 md5

執(zhí)行g(shù)pstop -u使配置生效