Ceph和Owncloud部署與使用
1.部署ceph集群
(1)虛擬機基礎配置
在VMware中設置VMnet1的網(wǎng)絡為僅主機模式驶拱,網(wǎng)段設置為203.0.113.0图毕;VMnet8設置為NAT模式腺兴,網(wǎng)段設置為10.0.0.0方咆,如圖7-11所示蔫慧。
圖7-11 虛擬機配置
(2)集群網(wǎng)絡配置
在三臺虛擬機ceph-1、ceph-2重挑、ceph-3上部署Ceph集群,三臺虛擬機的IP地址為:10.0.0.101棠涮、10.0.0.102谬哀、10.0.0.103,子網(wǎng)掩碼為255.255.255.0严肪,默認網(wǎng)關(guān)為10.0.0.2史煎,DNS服務器為10.0.0.2。
(3)配置client虛擬機
在VMware中創(chuàng)建一臺虛擬機驳糯,操作系統(tǒng)為CentOS-7-x86_64-DVD-1908篇梭,硬盤大小為20G,如圖7-12所示酝枢。
圖7-12 client虛擬機設置
(4)client網(wǎng)絡配置
為虛擬機配置主機名:client恬偷。配置IP地址:10.0.0.100,子網(wǎng)掩碼為255.255.255.0帘睦,默認網(wǎng)關(guān)為10.0.0.2袍患,DNS服務器為10.0.0.2,使虛擬機可以訪問Internet竣付。
2.配置Ceph對象存儲
(1)部署Ceph RGW
在ceph-1節(jié)點部署Ceph
RGW诡延。
[root@ceph-1 ~]# cd /opt/osd
[root@ceph-1 osd]# ceph-deploy rgwcreate ceph-1
……
(2)編輯pool文件
[root@ceph-1 osd]# vi /root/pool
.rgw
.rgw.root
.rgw.control
.rgw.gc
.rgw.buckets
.rgw.buckets.index
.rgw.buckets.extra
.log
.intent-log
.usage
.users
.users.email
.users.swift
.users.uid
(3)創(chuàng)建腳本
編輯創(chuàng)建和配置pool的腳本文件。
[root@ceph-1 osd]# vi/root/create_pool.sh
#!/bin/bash
PG_NUM=8
PGP_NUM=8
SIZE=3
for i in `cat /root/pool`
??????? do
??????? ceph osd pool create $i $PG_NUM
??????? ceph osd pool set $i size $SIZE
??????? done
for i in `cat /root/pool`
??????? do
??????? ceph osd pool set $i pgp_num $PGP_NUM
??????? done
(4)運行腳本
運行腳本文件古胆,創(chuàng)建對象存儲所使用的所有pool孕暇。
[root@ceph-1 osd]# chmod +x/root/create_pool.sh
[root@ceph-1 osd]#/root/create_pool.sh
pool '.rgw' created
set pool 5 size to 3
pool '.rgw.root' already exists
set pool 1 size to 3
pool '.rgw.control' created
set pool 6 size to 3
pool '.rgw.gc' created
set pool 7 size to 3
pool '.rgw.buckets' created
set pool 8 size to 3
pool '.rgw.buckets.index' created
set pool 9 size to 3
pool '.rgw.buckets.extra' created
set pool 10 size to 3
pool '.log' created
set pool 11 size to 3
pool '.intent-log' created
set pool 12 size to 3
pool '.usage' created
set pool 13 size to 3
pool '.users' created
set pool 14 size to 3
pool '.users.email' created
set pool 15 size to 3
pool '.users.swift' created
set pool 16 size to 3
pool '.users.uid' created
set pool 17 size to 3
set pool 5 pgp_num to 8
set pool 1 pgp_num to 8
set pool 6 pgp_num to 8
set pool 7 pgp_num to 8
set pool 8 pgp_num to 8
set pool 9 pgp_num to 8
set pool 10 pgp_num to 8
set pool 11 pgp_num to 8
set pool 12 pgp_num to 8
set pool 13 pgp_num to 8
set pool 14 pgp_num to 8
set pool 15 pgp_num to 8
set pool 16 pgp_num to 8
set pool 17 pgp_num to 8
(5)測試訪問集群
在ceph-1節(jié)點測試是否能訪問Ceph集群,測試成功會出現(xiàn)HEALTH_OK的字樣。
[root@ceph-1 osd]# cp/var/lib/ceph/radosgw/ceph-rgw.ceph-1/keyring/etc/ceph/ceph.client.rgw.ceph-1.keyring
[root@ceph-1 osd]# ceph -s -k/var/lib/ceph/radosgw/ceph-rgw.ceph-1/keyring --name client.rgw.ceph-1
?cluster:
???id:?? ??68ecba50-862d-482e-afe2-f95961ec3323
???health: HEALTH_OK
?services:
???mon: 3 daemons, quorum ceph-1,ceph-2,ceph-3 (age 21m)
???mgr: ceph-1(active, since 21m)
???osd: 3 osds: 3 up (since 21m), 3 in (since 7d)
???rgw: 1 daemon active (ceph-1)
??data:
???pools:?? 17 pools, 136 pgs
???objects: 187 objects, 1.2 KiB
???usage:?? 3.0 GiB used, 294 GiB /297 GiB avail
pgs:???? 136 active+clean
3.使用S3
API訪問Ceph對象存儲
(1)創(chuàng)建用戶
在ceph-1節(jié)點創(chuàng)建radosgw用戶妖滔。
[root@ceph-1 osd]# radosgw-adminuser create --uid=radosgw --display-name="radosgw"
{
???"user_id": "radosgw",
???"display_name": "radosgw",
???"email": "",
???"suspended": 0,
???"max_buckets": 1000,
???"subusers": [],
???"keys": [
??????? {
??????????? "user":"radosgw",
??????????? "access_key": "TMDWQAB2U9LDTHGID8UG",
??????????? "secret_key":"2z7cuRMxDzIZcOfycLR9eZs4dHF6N33gMVTd6ICp"
??????? }
???],
???"swift_keys": [],
???"caps": [],
???"op_mask": "read, write, delete",
???"default_placement": "",
???"default_storage_class": "",
???"placement_tags": [],
???"bucket_quota": {
??????? "enabled": false,
??????? "check_on_raw": false,
??????? "max_size": -1,
??????? "max_size_kb": 0,
??????? "max_objects": -1
???},
???"user_quota": {
??????? "enabled": false,
??????? "check_on_raw": false,
???????"max_size": -1,
??????? "max_size_kb": 0,
??????? "max_objects": -1
???},
???"temp_url_keys": [],
???"type": "rgw",
???"mfa_ids": []
}
(2)安裝bind服務
在client節(jié)點安裝bind服務隧哮。
[root@client ~]# mkdir /opt/bak
[root@client ~]# cd/etc/yum.repos.d
[root@client yum.repos.d]# mv */opt/bak
將CentOS7-Base-163.repo通過SFTP復制到client節(jié)點的/etc/yum.repos.d目錄中。
[root@client yum.repos.d]# ls
CentOS7-Base-163.repo
[root@client yum.repos.d]# yumclean all
[root@client yum.repos.d]# yummakecache
[root@client yum.repos.d]# yum -y installbind
(3)編輯bind主配置文件
[root@client ~]# vi /etc/named.conf
修改以下配置:
listen-on port 53 {127.0.0.1;10.0.0.100; };
allow-query???? { localhost;10.0.0.0/24; };
添加以下配置:
zone "lab.net" IN {
??????? type master;
??????? file "db.lab.net";
??????? allow-update { none; };
};
(4)編輯域lab.net的區(qū)域配置文件
[root@client ~]# vi/var/named/db.lab.net
@ 86400 IN SOA lab.net.root.lab.net. (
??????? 20191120
??????? 10800
??????? 3600
??????? 3600000
??????? 86400 )
@ 86400 IN NS lab.net.
@ 86400 IN A 10.0.0.101
* 86400 IN CNAME @
(5)檢查配置文件
[root@client ~]# named-checkconf/etc/named.conf
[root@client ~]# named-checkzonelab.net /var/named/db.lab.net
zone lab.net/IN: loaded serial20191120
OK
(6)啟動bind服務
[root@client ~]# systemctl startnamed
[root@client ~]# systemctl enable named
Created symlink from/etc/systemd/system/multi-user.target.wants/named.service to/usr/lib/systemd/system/named.service.
(7)配置DNS服務
編輯網(wǎng)卡配置文件座舍,將DNS服務器指向client自己的IP地址沮翔。
[root@client ~]# vi/etc/sysconfig/network-scripts/ifcfg-ens32
DNS1=10.0.0.100
(8)定義DNS地址
編輯/etc/resolv.conf,將DNS服務器指向client自己的IP地址曲秉。
[root@client ~]# vi/etc/resolv.conf
nameserver 10.0.0.100
(9)安裝nslookup
安裝nslookup采蚀,測試DNS配置
[root@client ~]# yum -y installbind-utils
[root@client ~]# nslookup
> ceph-1.lab.net
Server:???????? 10.0.0.100
Address:??????? 10.0.0.100#53
ceph-1.lab.net? canonical name = lab.net.
Name:?? lab.net
Address: 10.0.0.101
> exit
(10)安裝s3cmd
訪問https://s3tools.org/download,下載s3cmd的2.0.2版本承二。
[root@client ~]# ls
anaconda-ks.cfg? s3cmd-2.0.2.zip
[root@client ~]# yum -y installunzip python-dateutil
[root@client ~]# unzips3cmd-2.0.2.zip
……
(11)配置s3cmd
[root@client ~]# cd s3cmd-2.0.2
[root@client s3cmd-2.0.2]# ./s3cmd--configure
Enter new values or accept defaultsin brackets with Enter.
Refer to user manual for detaileddescription of all options.
Access key and Secret key are youridentifiers for Amazon S3. Leave them empty for using the env variables.
Access Key: TMDWQAB2U9LDTHGID8UG
Secret Key:2z7cuRMxDzIZcOfycLR9eZs4dHF6N33gMVTd6ICp
Default Region [US]:
Use "s3.amazonaws.com"for S3 Endpoint and not modify it to the target Amazon S3.
S3 Endpoint [s3.amazonaws.com]:ceph-1.lab.net:7480
Use"%(bucket)s.s3.amazonaws.com" to the target Amazon S3."%(bucket)s" and "%(location)s" vars can be used
if the target S3 system supportsdns based buckets.
DNS-style bucket+hostname:porttemplate for accessing a bucket [%(bucket)s.s3.amazonaws.com]:%(bucket).ceph-1.lab.net:7480
Encryption password is used toprotect your files from reading
by unauthorized persons while intransfer to S3
Encryption password:
Path to GPG program [/usr/bin/gpg]:
When using secure HTTPS protocolall communication with Amazon S3
servers is protected from 3rd partyeavesdropping. This method is
slower than plain HTTP, and canonly be proxied with Python 2.7 or newer
Use HTTPS protocol [Yes]: no
On some networks all internetaccess must go through a HTTP proxy.
Try setting it here if you can'tconnect to S3 directly
HTTP Proxy server name:
New settings:
?Access Key: TMDWQAB2U9LDTHGID8UG
?Secret Key: 2z7cuRMxDzIZcOfycLR9eZs4dHF6N33gMVTd6ICp
?Default Region: US
?S3 Endpoint: ceph-1.lab.net:7480
?DNS-style bucket+hostname:port template for accessing a bucket:%(bucket).ceph-1.lab.net:7480
?Encryption password:
?Path to GPG program: /usr/bin/gpg
?Use HTTPS protocol: False
?HTTP Proxy server name:
?HTTP Proxy server port: 0
Test access with suppliedcredentials? [Y/n] n
Save settings? [y/N] y
Configuration saved to'/root/.s3cfg'
(12)顯示存儲桶
使用命令可以查看S3的存儲桶榆鼠,此處因為還沒有使用命令創(chuàng)建,所以結(jié)果為空亥鸠。
[root@client s3cmd-2.0.2]# ./s3cmdls
(13)創(chuàng)建存儲桶
在client節(jié)點可以使用命令創(chuàng)建存儲桶bucket
[root@client s3cmd-2.0.2]# ./s3cmdmb s3://bucket
Bucket 's3://bucket/' created
[root@client s3cmd-2.0.2]# ./s3cmdls
2019-11-23 07:45? s3://bucket
(14)上傳文件
將/etc/named.conf和/var/named/db.lab.net上傳到存儲桶bucket中
[root@client s3cmd-2.0.2]# ./s3cmdput /etc/named.conf s3://bucket
WARNING: Module python-magic is notavailable. Guessing MIME types based on file extensions.
upload: '/etc/named.conf' ->'s3://bucket/named.conf'? [1 of 1]
?1933 of 1933??100% in??? 1s? 1317.89 B/s?done
[root@client s3cmd-2.0.2]# ./s3cmdput /var/named/db.lab.net s3://bucket
WARNING: Module python-magic is notavailable. Guessing MIME types based on file extensions.
upload: '/var/named/db.lab.net'-> 's3://bucket/db.lab.net'? [1 of 1]
?182 of 182??100% in??? 0s???? 2.78 kB/s?done
4.部署LAMP環(huán)境
(1)安裝最新源
在client節(jié)點安裝epel和webtatic軟件源妆够,防止后面出現(xiàn)丟包。
[root@client ~]# rpm -Uvhhttps://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
[root@client ~]# rpm -Uvhhttps://mirror.webtatic.com/yum/el7/webtatic-release.rpm
[root@client ~]# yum makecache
(2)安裝LAMP
在client節(jié)點需要安裝Owncloud所需的LAMP環(huán)境軟件包负蚊。
[root@client ~]# yum -y install httpd php72wmariadb-server php72w-mysql php72w-gd php72w-xml php72w-intl php72w-mbstring
(3)啟動數(shù)據(jù)庫
在搭建Owncloud網(wǎng)盤服務的時候神妹,需要用到數(shù)據(jù)庫,在上一步已經(jīng)安裝了數(shù)據(jù)庫家妆,此處需要啟用MariaDB數(shù)據(jù)庫服務鸵荠。
[root@client ~]# systemctl start mariadb
[root@client ~]# systemctl enable mariadb
(4)啟動Apache服務
啟動和啟用Apache HTTPD Web服務
[root@client ~]# systemctl start httpd
[root@client ~]# systemctl enable httpd
(5)關(guān)閉防火墻
[root@client ~]# systemctl stop firewalld
[root@client ~]# systemctl disable firewalld
(6)關(guān)閉SElinux
將SELinux模式設置為permissive,并檢查是否設置完成
[root@client ~]# setenforce 0
[root@client ~]# vi /etc/selinux/config
SELINUX=permissive
5.配置Owncloud
(1)上傳Owncloud壓縮包
將OwnCloudTarball_10_0_9.zip通過SFTP傳輸?shù)絚lient節(jié)點的/root目錄伤极,并查看蛹找。
(2)解壓Owncloud
有些centos系統(tǒng)需要自己安裝unzip解壓文件,此處如果出現(xiàn)無法解壓時需看清報錯信息哨坪。
[root@client ~]# unzipOwnCloudTarball_10_0_9.zip
(3)更新web文件
將Owncloud文件復制到Web服務器默認網(wǎng)站的主目錄中庸疾,此處使用cp -rf命令是將當前目錄下所有文件復制過去。
[root@client ~]# cd owncloud-10.0.9
[root@client owncloud-10.0.9]# cp -rf */var/www/html
(4)設置目錄權(quán)限
[root@client owncloud-10.0.9]# cd/var/www/html
[root@client html]# chmod -R 777 config
[root@client html]# mkdir data
[root@client html]# chown -R apache:apachedata
(5)設置數(shù)據(jù)庫密碼
此處設置MariaDB數(shù)據(jù)庫root用戶的密碼為123456齿税,后面網(wǎng)盤服務連接數(shù)據(jù)庫時需要設置同樣密碼,不然無法提供后端存儲服務炊豪,會導致安裝失敗凌箕。
[root@client html]# mysqladmin -u rootpassword '123456'
(6)創(chuàng)建owncloud數(shù)據(jù)庫
登錄MariaDB數(shù)據(jù)庫服務器并創(chuàng)建owncloud數(shù)據(jù)庫,作為后端的數(shù)據(jù)存儲词渤。
[root@client html]# mysql -uroot -p123456
Welcome to the MariaDB monitor.? Commands end with ; or \g.
Your MariaDB connection id is 3
Server version: 5.5.64-MariaDB MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDBCorporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' toclear the current input statement.
MariaDB [(none)]> CREATE DATABASEowncloud;
Query OK, 1 row affected (0.00 sec)
MariaDB [(none)]> exit
Bye
(7)初始化owncloud網(wǎng)頁
在初始化owncloud的頁面中會讓我們進行一些設置牵舱,比如用戶、密碼和數(shù)據(jù)庫等缺虐。選數(shù)據(jù)庫的時候千萬要注意幾點:
[if !supportLists]n? [endif]首先數(shù)據(jù)庫類型要選擇mysql/mariadb芜壁,如果選的是sqlite的話,性能會不太好,后面在owncloud的設置頁面中也會有提示慧妄。到那時想改就比較麻煩了顷牌。
[if !supportLists]n? [endif]然后就是如果你的服務器原先就創(chuàng)建過用戶和密碼,那在選擇好數(shù)據(jù)庫類型后塞淹,下面的用戶和密碼就是你之前的用戶名和密碼
使用瀏覽器訪問client節(jié)點的IP地址窟蓝,創(chuàng)建管理員賬號admin,為admin用戶設置密碼饱普,配置數(shù)據(jù)庫使用MySQL/MariaDB运挫,如圖7-13所示。
圖7-13 owncloud初始化界面
(8)安裝owncloud
輸入數(shù)據(jù)庫用戶名root套耕、密碼123456谁帕、數(shù)據(jù)庫名owncloud,單擊安裝完成冯袍,如圖7-14所示匈挖。
圖7-14 owncloud安裝
(9)登陸owncloud
使用admin用戶登錄Owncloud,如圖7-15所示颠猴。
圖7-15 owncloud登陸
如果登錄出現(xiàn)錯誤关划,在client節(jié)點輸入以下命令,設置php文件的權(quán)限為可讀可寫可執(zhí)行翘瓮。
# chmod -R 777 /var/lib/php/session
(10)配置外部存儲
單擊頁面右上角的adminà設置à管理à存儲贮折,單擊啟用Enable external storage。輸入目錄名稱為s3资盅,選擇外部存儲為Amazon S3调榄,輸入存儲桶為bucket、主機名為ceph-1.lab.net呵扛、端口號為7480每庆、區(qū)域為US,啟用Path Style今穿,輸入Access Key和Secret Key缤灵。當看到前面出現(xiàn)綠色圓圈時,表示外部存儲配置成功蓝晒,如圖7-16所示腮出。
圖7-16 配置外部存儲
(11)查看S3目錄
通過主界面可以看到之前創(chuàng)建的s3目錄,如圖7-17所示
圖7-17 S3外部存儲
(12)網(wǎng)盤測試
此時雙擊s3可以進入s3目錄芝薇,并且可以看到之前實驗使用s3cmd上傳的文件胚嘲,也可以在Web界面上傳文件或下載文件,還原了平時所使用網(wǎng)盤的功能洛二,如圖7-18所示馋劈。
圖7-18 網(wǎng)盤測試