簡(jiǎn)介
- FastDFS是一個(gè)開源的輕量級(jí)分布式文件系統(tǒng)骆姐，它對(duì)文件進(jìn)行管理镜粤，功能包括：文件存儲(chǔ)、文件同步诲锹、文件訪問(wèn)（文件上傳繁仁、文件下載）等，解決了大容量存儲(chǔ)和負(fù)載均衡的問(wèn)題归园。FastDFS服務(wù)端有兩個(gè)角色：跟蹤器（tracker）和存儲(chǔ)節(jié)點(diǎn)（storage）黄虱。跟蹤器主要做調(diào)度工作，在訪問(wèn)上起負(fù)載均衡的作用庸诱。
-
存儲(chǔ)節(jié)點(diǎn)存儲(chǔ)文件捻浦，完成文件管理的所有功能：就是這樣的存儲(chǔ)晤揣、同步和提供存取接口，F(xiàn)astDFS同時(shí)對(duì)文件的metadata進(jìn)行管理朱灿。所謂文件的meta data就是文件的相關(guān)屬性昧识，以鍵值對(duì)（key valuepair）方式表示，如：width=1024盗扒，其中的key為width跪楞，value為1024。文件metadata是文件屬性列表侣灶，可以包含多個(gè)鍵值對(duì)甸祭。
拓?fù)鋱D
部署
- 環(huán)境安裝
yum install -y zlib zlib-devel pcre pcre-devel gcc gcc-c++ openssl openssl-devel libevent libevent-devel perl unzip 或者 yum groupinstall 'Development tools' -y
- 下載軟件包
下載Libfastcommon
wget https://github.com/happyfish100/libfastcommon/archive/master.zip
下載FastDFS
wget https://github.com/happyfish100/fastdfs/archive/V5.05.tar.gz
下載FastDFS-Nginx模塊
wget http://jaist.dl.sourceforge.net/project/fastdfs/FastDFS%20Nginx%20Module%20Source%20Code/fastdfs-nginx-module_v1.16.tar.gz
下載Nginx
wget http://nginx.org/download/nginx-1.8.0.tar.gz
- 創(chuàng)建數(shù)據(jù)儲(chǔ)存目錄
#在tracker_server上創(chuàng)建tracker目錄；在storage_server上創(chuàng)建storage目錄
mkdir -p /data/fastdfs/{storage,tracker}
- 安裝Libfastcommon
Tracker-server和storage-server都需要安裝
unzip libfastcommon-master.zip
cd libfastcommon-master
./make.sh
./make.sh install
ln -s /usr/lib64/libfastcommon.so /usr/local/lib/libfastcommon.so
ln -s /usr/lib64/libfastcommon.so /usr/lib/libfastcommon.so
ln -s /usr/lib64/libfdfsclient.so /usr/local/lib/libfdfsclient.so
ln -s /usr/lib64/libfdfsclient.so /usr/lib/libfdfsclient.so
- 安裝FastDFS
解壓褥影、編譯安裝FastDFS
tar -xf FastDFS_v5.08.tar.gz
cd FastDFS
sh make.sh && sh make.sh install
cp -pa conf/*.conf /etc/fdfs/
cp -pa conf/mime.types /etc/fdfs/
cd /etc/fdfs/
rm -rf *.sample
chown -R nobody.nobody /data/fastdfs
chown -R nobody.nobody /etc/fdfs
注：這里賦予nobody賬號(hào)權(quán)限是因?yàn)閚ginx用nobody用戶啟動(dòng)
- 修改tracker.conf配置
disabled=false
bind_addr=10.12.8.42
port=22122
connect_timeout=30
network_timeout=60
base_path=/data/fastdfs/tracker
max_connections=256
accept_threads=1
work_threads=4
store_lookup=2
store_group=group1
store_server=0
store_path=0
download_server=0
reserved_storage_space = 10%
log_level=info
run_by_group=
run_by_user=
allow_hosts=*
sync_log_buff_interval = 10
check_active_interval = 120
thread_stack_size = 64KB
storage_ip_changed_auto_adjust = true
storage_sync_file_max_delay = 86400
storage_sync_file_max_time = 300
use_trunk_file = false
slot_min_size = 256
slot_max_size = 16MB
trunk_file_size = 64MB
trunk_create_file_advance = false
trunk_create_file_time_base = 02:00
trunk_create_file_interval = 86400
trunk_create_file_space_threshold = 20G
trunk_init_check_occupying = false
trunk_init_reload_from_binlog = false
trunk_compress_binlog_min_interval = 0
use_storage_id = false
storage_ids_filename = storage_ids.conf
id_type_in_filename = ip
store_slave_file_use_link = false
rotate_error_log = false
error_log_rotate_time=00:00
rotate_error_log_size = 0
log_file_keep_days = 0
use_connection_pool = false
connection_pool_max_idle_time = 3600
http.server_port=80
http.check_alive_interval=30
http.check_alive_type=tcp
http.check_alive_uri=/status.html
- 修改storage.conf配置
disabled=false
group_name=group1(這里storage屬于group1,如有多個(gè)group根據(jù)實(shí)際情況填寫)
bind_addr=
client_bind=true
port=23000
connect_timeout=30
network_timeout=60
heart_beat_interval=30
stat_report_interval=60
base_path=/data/fastdfs/storage
max_connections=256
buff_size = 256KB
accept_threads=1
work_threads=4
disk_rw_separated = true
disk_reader_threads = 1
disk_writer_threads = 1
sync_wait_msec=50
sync_interval=0
sync_start_time=00:00
sync_end_time=23:59
write_mark_file_freq=500
store_path_count=1
store_path0=/data/fastdfs/storage
subdir_count_per_path=256
tracker_server=10.12.8.42:22122(tracker地址,可寫多個(gè))
tracker_server=10.12.8.44:22122(tracker地址,可寫多個(gè))
log_level=info
run_by_group=
run_by_user=
allow_hosts=*
file_distribute_path_mode=0
file_distribute_rotate_count=100
fsync_after_written_bytes=0
sync_log_buff_interval=10
sync_binlog_buff_interval=10
sync_stat_file_interval=300
thread_stack_size=512KB
upload_priority=10
if_alias_prefix=
check_file_duplicate=0
file_signature_method=hash
key_namespace=FastDFS
keep_alive=0
use_access_log = false
rotate_access_log = false
access_log_rotate_time=00:00
rotate_error_log = false
error_log_rotate_time=00:00
rotate_access_log_size = 0
rotate_error_log_size = 0
log_file_keep_days = 0
file_sync_skip_invalid_record=false
use_connection_pool = false
connection_pool_max_idle_time = 3600
http.domain_name=
http.server_port=80
- 修改mod_fastdfs.conf配置
connect_timeout=2
network_timeout=30
base_path=/data/fastdfs/storage
load_fdfs_parameters_from_tracker=true
storage_sync_file_max_delay = 86400
use_storage_id = false
storage_ids_filename = storage_ids.conf
tracker_server=10.12.8.42:22122
storage_server_port=23000
group_name=group1
url_have_group_name = true
store_path_count=1
store_path0=/data/fastdfs/storage
log_level=info
log_filename=
response_mode=proxy
if_alias_prefix=
flv_support = true
flv_extension = flv
group_count = 0
- 安裝Nginx和Fastdfs-Nginx模塊
tar -zxf fastdfs-nginx-module_v1.16.tar.gz && tar -zxf nginx-1.10.1.tar.gz
cd fastdfs-nginx-module/src/
cp -r mod_fastdfs.conf /etc/fdfs/
vim /soft/FastDFS/fastdfs-nginx-module/src/config
#修改config文件如下
更改前：“CORE_INCS="$CORE_INCS /usr/local/include/fastdfs /usr/local/include/fastcommon/"
CORE_LIBS="$CORE_LIBS -L/usr/local/lib -lfastcommon -lfdfsclient"”
更改后：“CORE_INCS="$CORE_INCS /usr/include/fastdfs /usr/local/include/fastcommon/"
CORE_LIBS="$CORE_LIBS -L/usr/lib64 -lfastcommon -lfdfsclient"”
或者
ln -sv /usr/incloud/fastcommon /usr/local/incloud/fastcommon
ln -sv /usr/incloud/fastdfs /usr/local/incloud/fastdfs
ln -sv /usr/lib64/libfastcommon.so /usr/local/lib/libfastcommon.so
#安裝nginx
cd nginx-1.10.1
./configure --add-module=/soft/FastDFS/fastdfs-nginx-module/src/ --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_ssl_module --with-http_realip_module
make && make install
-
vhost/*.conf配置
server {
listen 80 default;
server_name 10.12.8.42;
access_log logs/access.v5ehomepay.log main;
error_log logs/error.v5ehomepay.log;
location /nginx-status {
stub_status on;
access_log on;
}
location /group1/M00 {
root /data/fastdfs/storage/data/;
ngx_fastdfs_module;
}
location ~ .*\.(gif|jpg|jpeg|png|bmp|swf|js)$ {
# fastcgi_cache cache_one; #nginx.conf 開啟cache才行池户，要不然啟動(dòng)報(bào)錯(cuò)
fastcgi_cache_valid 200 10m;
fastcgi_cache_valid 304 3m;
fastcgi_cache_valid 301 302 1h;
fastcgi_cache_valid any 1m;
fastcgi_cache_min_uses 1;
fastcgi_cache_use_stale error timeout invalid_header http_500;
fastcgi_cache_key $host$request_uri;
access_log off;
}
error_page 403 404 /40x.html;
location = /40x.html root html;
}
error_page 404 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
- 啟動(dòng)進(jìn)程，并查看端口
/etc/init.d/fdfs_trackerd start
/etc/init.d/fdfs_storaged start
Starting FastDFS storage server:
/usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf
#查看進(jìn)程
netstat -npl|grep -E "nginx|fdfs"
tcp 0 0 0.0.0.0:23000 0.0.0.0:* LISTEN 27454/fdfs_storaged
tcp 0 0 10.12.8.42:22122 0.0.0.0:* LISTEN 14179/fdfs_trackerd
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 14201/nginx
ps:注意事情：Centos 7.1版本中trackerd和storaged啟動(dòng)失敗時(shí)凡怎，做以下操作
ln -s /usr/bin/fdfs_trackerd /usr/local/bin
ln -s /usr/bin/fdfs_storaged /usr/local/bin
ln -s /usr/bin/stop.sh /usr/local/bin
ln -s /usr/bin/restart.sh /usr/local/bin
- 測(cè)試
echo 'test fastDFS!!!' > test.htm
vim /etc/fdfs/client.conf
添加tracker_server地址
fdfs_test /etc/fdfs/client.conf upload test.html
This is FastDFS client test program v5.08
Copyright (C) 2008, Happy Fish / YuQing
FastDFS may be copied only under the terms of the GNU General
Public License V3, which may be found in the FastDFS source kit.
Please visit the FastDFS Home Page http://www.csource.org/
for more detail.
[2016-08-03 15:40:38] DEBUG - base_path=/data/fastdfs/storage, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0
tracker_query_storage_store_list_without_group:
server 1. group_name=, ip_addr=10.12.8.42, port=23000
server 2. group_name=, ip_addr=10.12.8.43, port=23000
group_name=group1, ip_addr=10.12.8.43, port=23000
storage_upload_by_filename
group_name=group1, remote_filename=M00/00/00/CgwIK1ehn_aATP3dAAAAELIjPeQ51.html
source ip address: 10.12.8.43
file timestamp=2016-08-03 15:40:38
file size=16
file crc32=2988654052
example file url: http://10.12.8.43/group1/M00/00/00/CgwIK1ehn_aATP3dAAAAELIjPeQ51.html
storage_upload_slave_by_filename
group_name=group1, remote_filename=M00/00/00/CgwIK1ehn_aATP3dAAAAELIjPeQ51_big.html
source ip address: 10.12.8.43
file timestamp=2016-08-03 15:40:38
file size=16
file crc32=2988654052
example file url: http://10.12.8.43/group1/M00/00/00/CgwIK1ehn_aATP3dAAAAELIjPeQ51_big.html
curl http://10.12.8.43/group1/M00/00/00/CgwIK1ehn_aATP3dAAAAELIjPeQ51_big.html
test fastDFS!!!
