1喂江、haproxy https實現(xiàn)
- 配置haproxy哨鸭，執(zhí)行SSL加解密
[root@centos8mini ~]# cat /etc/haproxy/haproxy.cfg
...
frontend www
log global
option httplog
bind 192.168.156.11:80
#需要加上ssl選項
bind 192.168.156.11:443 ssl crt /root/mxx.pem
#http到https重定向
redirect scheme https if ! { ssl_fc }
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { ssl_fc }
use_backend aaa
mode http
#capture request header Host len 256
#capture request header User-Agent len 512
#capture request header Referer len 15
#capture request header X-Forwarded-For len 15
backend aaa
cookie Dserver insert nocache indirect
server rs1 192.168.156.202:80 cookie web1 send-proxy
server rs2 192.168.156.204:80 cookie web2 send-proxy
...
- 配置nginx日志淮阐，記錄x-forwarded-port和x-forwarded-proto信息：
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for" "$host" "$proxy_protocol_addr" "$http_x_forwarded_port" "$http_x_forwarded_proto"';
- 測試結果
#后端服務器可以看到https爆班，haproxy加上去的請求頭部字段
192.168.156.208 - - [21/Jan/2022:21:11:36 +0800] "GET / HTTP/1.1" 200 110 "-" "curl/7.58.0" "192.168.156.100" "www.mxx.com" "192.168.156.100" "443" "https"
- 抓包可以看到前端是80切443南吮，后端一直是80花嘶，ssl的加解密由haproxy完成
#客戶端100去訪問VIP 156.11的443端口
21:24:23.882974 IP 192.168.156.11.443 > 192.168.156.100.37330: Flags [P.], seq 1353:1624, ack 695, win 235, options [nop,nop,TS val 3578864071 ecr 3498970570], length 271
21:24:23.883118 IP 192.168.156.11.443 > 192.168.156.100.37330: Flags [P.], seq 1624:1895, ack 695, win 235, options [nop,nop,TS val 3578864071 ecr 3498970570], length 271
21:24:23.883393 IP 192.168.156.100.37330 > 192.168.156.11.443: Flags [.], ack 1895, win 501, options [nop,nop,TS val 3498970571 ecr 3578864071], length 0
#haproxy 208去訪問后端的204的80端口
21:24:23.883424 IP 192.168.156.208.46246 > 192.168.156.204.80: Flags [S], seq 3363885263, win 29200, options [mss 1460,sackOK,TS val 3269997331 ecr 0,nop,wscale 7], length 0
21:24:23.883460 IP 192.168.156.204.80 > 192.168.156.208.46246: Flags [S.], seq 3613891712, ack 3363885264, win 28960, options [mss 1460,sackOK,TS val 1974460156 ecr 3269997331,nop,wscale 7], length 0
21:24:23.884099 IP 192.168.156.208.46246 > 192.168.156.204.80: Flags [.], ack 1, win 229, options [nop,nop,TS val 3269997332 ecr 1974460156], length 0
21:24:23.884167 IP 192.168.156.208.46246 > 192.168.156.204.80: Flags [P.], seq 1:231, ack 1, win 229, options [nop,nop,TS val 3269997332 ecr 1974460156], length 230: HTTP
21:24:23.884177 IP 192.168.156.204.80 > 192.168.156.208.46246: Flags [.], ack 231, win 235, options [nop,nop,TS val 1974460157 ecr 3269997332], length 0
2笋籽、總結tomcat的核心組件以及根目錄結構
tomcat核心組件
- Server組件：一個server就是一個tomcat實例，不同server使用不同的端口
- service組件：包含了Engine和Connector的對應關系
- Connector組件：可以簡單理解為一個監(jiān)聽端口椭员，包含HTTP车海、HTTPS、AJP協(xié)議的連接器
- 容器類組件：在Engine內創(chuàng)建多個Host隘击，Host內可以配置本地磁盤路徑和URL路徑的關聯(lián)關系
- Engine：接收用戶通過某個接口發(fā)送的處理請求容劳；Server里只有一個Engine
- Host：一個Engine可以包含多個虛擬主機，和Apache闸度、Nginx的虛擬主機是一樣的概念竭贩，通過端口、IP莺禁、域名等方式區(qū)分虛擬主機
- Context：執(zhí)行路徑映射留量，類似Apache或Nginx里的Location，配置磁盤目錄和URL路徑的關聯(lián)關系
- 內嵌類：可以內嵌到其他組件內，如valve楼熄、logger忆绰、realm、loader可岂、manager等
- 集群類組件：listener错敢、cluster
根目錄結構
- bin/：包含服務啟動、停止的程序和文件缕粹，如shutdown.sh稚茅、startup.sh、catalina.sh等
- conf/：包含全局配置文件平斩，比如啟動service時亚享，EnvironmentFile=tomcat.conf文件、web.xml绘面、server.xml欺税、context.xml、tomcat-users.xml等
- lib/：java庫目錄
- logs/：日志文件的存放位置
- webapps/：類似apache的/var/www/html揭璃，是tomcat默認頁面的存放路徑
- work/：jsp編譯后的字節(jié)碼和servlet文件存放位置晚凿，通常jsp文件需要被提前預熱訪問，以加快用戶的訪問速度瘦馍；
- 該目錄中的緩存文件可能會影響新版本資源的發(fā)布晃虫，需要刪除這些自動生成的文件，讓新版本的jsp可以重新生成
3扣墩、tomcat實現(xiàn)多虛擬主機
#!/bin/bash
#****************************************************************************************#
#Author: Yabao11
#QQ: what QQ，no QQ
#Date: 2022-01-04
#FileName: nginx.sh
#URL: https://github.com/yabao11
#Description: Test Script
#Copyright (C): 2022 All rights reserved
#*******************************定義顏色*************************************************#
RED="\e[1;31m"
GREEN="\e[1;32m"
SKYBLUE="\e[1;36m"
YELLOW="\e[1;43m"
BLUE="\e[1;44m"
END="\e[0m"
RandomColor="\e[1;32m"
#****************************************************************************************#
function Ostype {
if grep -i -q "release 6" /etc/centos-release;then
echo Centos6
elif grep -i -q Centos-8 /etc/os-release;then
echo Centos
elif grep -i -q Centos-7 /etc/os-release;then
echo Centos7
elif grep -i -q Ubuntu /etc/os-release;then
echo Ubuntu
elif grep -i -q "RedHat" /etc/os-release;then
echo Redhat
fi
}
function color {
RES_COL=60
MOVE_TO_COL="echo -en \E[${RES_COL}G"
SETCOLOR_SUCCESS="echo -en \E[1;32m"
SETCOLOR_FAILURE="echo -en \E[1;31m"
SETCOLOR_WARNING="echo -en \E[1;33m"
SETCOLOR_NORMAL="echo -en \E[0m"
echo -n "$1" && $MOVE_TO_COL
echo -n "["
if [[ $2 = "success" || $2 = "0" ]]; then
${SETCOLOR_SUCCESS}
echo -n " OK "
elif [[ $2 = "failure" || $2 = "1" ]]; then
${SETCOLOR_FAILURE}
echo -n "FAILED"
else
${SETCOLOR_WARNING}
echo -n "WARNING"
fi
${SETCOLOR_NORMAL}
echo -n "]"
echo
}
function jdk_install {
read -p "輸入希望二進制安裝的jdk版本扛吞，直接回車默認安裝'/root/jdk-8u321-linux-x64'" jdk
oracle_jdk=${jdk:-jdk-8u321-linux-x64}
[ -e /root/${oracle_jdk}.tar.gz ] && tar xvf /root/${oracle_jdk}.tar.gz -C /usr/local/src/ || { color "文件不存在" 2;exit; }
mv /usr/local/src/jdk* /usr/local/src/jdk
cat > /etc/profile.d/jdk.sh <<\EOF
export JAVA_HOME=/usr/local/src/jdk
export PATH=$PATH:$JAVA_HOME/bin
export JRE_HOME=$JAVA_HOME/jre
export CLASSPATH=$JAVA_HOME/lib/:$JRE_HOME/lib/
EOF
. /etc/profile.d/jdk.sh
java -version > /dev/null && color "jdk安裝成功呻惕！" 0
}
read -p "輸入希望二進制安裝的tomcat版本，直接回車默認安裝'apache-tomcat-8.5.75'" tomcat
read -p "輸入JAVA_HOME路徑滥比，直接回車使用默認路徑：'/usr/local/src/jdk'" pjdk
read -p "輸入tomcat路徑亚脆，直接回車使用默認路徑：'/usr/local/src/tomcat'" ptomcat
tomcat_version=${tomcat:-apache-tomcat-8.5.75}
java_path=${pjdk:-/usr/local/src/jdk}
tomcat_path=${ptomcat:-/usr/local/src/tomcat}
function tomcat_install {
java -version > /dev/null || { color "缺少java環(huán)境！" 1;exit; }
[ -e /root/${tomcat_version}.tar.gz ] || { yum -y install wget;wget https://mirrors.tuna.tsinghua.edu.cn/apache/tomcat/tomcat-8/v8.5.75/bin/${tomcat_version}.tar.gz; }
[ -e /root/${tomcat_version}.tar.gz ] || { color "找不到tomcat文件" 1;exit; }
tar xf ${tomcat_version}.tar.gz -C /usr/local/src/
[ -e ${tomcat_path} ] && rm -rf ${tomcat_path}
ln -s /usr/local/src/${tomcat_version} ${tomcat_path}
echo 'PATH=${tomcat_path}/bin:$PATH' > /etc/profile.d/tomcat.sh
. /etc/profile.d/tomcat.sh
useradd -r -s /sbin/nologin tomcat
chmod +x ${tomcat_path}/bin/*
catalina.sh version > /dev/null && color "安裝成功盲泛！" 0 || color "catalina命令執(zhí)行失敗濒持，請檢查權限或路徑" 1
cat > ${tomcat_path}/conf/tomcat.conf <<EOF
JAVA_HOME=${java_path}
CATALINA_HOME=${tomcat_path}
EOF
chown -R tomcat.tomcat ${tomcat_path}/ && color "tomcat目錄權限修改成功" 0 || color "tomcat目錄權限修改失敗" 1
echo -e $GREEN"創(chuàng)建tomcat.service文件"$END
[ -e /lib/systemd/system/tomcat.service ] || cat > /lib/systemd/system/tomcat.service <<EOF
[Unit]
Description=Tomcat
#After=syslog.target network.target remote-fs.target nss-lookup.target
After=syslog.target network.target
[Service]
Type=forking
EnvironmentFile=${tomcat_path}/conf/tomcat.conf
ExecStart=${tomcat_path}/bin/startup.sh
ExecStop=${tomcat_path}/bin/shutdown.sh
PrivateTmp=true
User=tomcat
Group=tomcat
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable --now tomcat && color "tomcat啟動成功！" 0 || color "tomcat啟動失敗寺滚，請執(zhí)行journalctl -u tomcat查看日志詳情" 1
}
function tomcat_config {
sed -i.bak -r -e 's|(pattern.*)("[[:space:]]+\/>)|\1 %{JSESSIONID}c %{HOST}i %{X-Forwarded-For}i %{X-Via}o\2|' \
-e 's|shutdown="SHUTDOWN"|shutdown="44ba3c71d57f494992641b258b965f28"|' \
-e '/<\/Engine>/i <Host name="www.mxx.com" appBase="mxx"\n unpackWARs="true" autoDeploy="true">\n <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"\n prefix="mxx_access_log" suffix=".txt"\n pattern="%h %l %u %t "%r" %s %b %{JSESSIONID}c %{HOST}i %{X-Forwarded-For}i %{X-Via}o" />\n </Host>\n' \
${tomcat_path}/conf/server.xml && color "配置日志格式柑营，修改server參數(shù)" 0
sed -i.bak -r '/<\/tomcat-users>/i<role rolename="manager-gui"/>\n<role rolename="admin-gui"/>\n<user username="tomcat" password="tomcat" roles="manager-gui,admin-gui"/>\n' ${tomcat_path}/conf/tomcat-users.xml && color "配置允許訪問tomcat的status等管理頁面" 0
sed -i.bak -r 's|(allow.*)("[[:space:]]+\/>)|\1\|\\d+\\.\\d+\\.\\d+\\.\\d+\2|' ${tomcat_path}/webapps/manager/META-INF/context.xml && color "配置允許所有IP地址訪問status（測試環(huán)境）" 0
sed -i.bak -r 's|(allow.*)("[[:space:]]+\/>)|\1\|\\d+\\.\\d+\\.\\d+\\.\\d+\2|' ${tomcat_path}/webapps/host-manager/META-INF/context.xml && color "配置允許所有IP地址訪問host-manager（測試環(huán)境）" 0
echo -e $GREEN"創(chuàng)建mxx虛擬主機"$END
mkdir ${tomcat_path}/mxx/ROOT -p
chown tomcat.tomcat ${tomcat_path}/mxx -R
cat > ${tomcat_path}/mxx/ROOT/index.jsp <<\EOF
<%@ page import="java.util.*" %>
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" http-equiv=Content-Type content="text/html;charset=utf-8">
<title>tomcat test</title>
</head>
<body>
<h1> Tomcat Website </h1>
<div>On <%=request.getServerName() %></div>
<div><%=request.getLocalAddr() + ":" + request.getLocalPort() %></div>
<div>SessionID = <span style="color:blue"><%=session.getId() %></span></div>
<%=new Date()%>
<h1>這是一個測試頁面</h1>
</body>
</html>
EOF
echo -e $GREEN"創(chuàng)建多個虛擬主機"$END
while true;do
read -p "是否需要繼續(xù)創(chuàng)建虛擬主機？（yes or no）" askuser1
askuser1=`echo $askuser1 | tr 'A-Z' 'a-z'`
case $askuser1 in
y|yes)
let i=++i
;;
n|no)
break
;;
*)
inputerror
;;
esac
read -p "輸入虛擬主機的域名" vhost
if [ -z ${vhost} ];then
while [ -z ${vhost} ];do
read -p "請輸入虛擬主機的域名" vhost
done
fi
sed -i.bak -r -e '/<\/Engine>/i <Host name="'${vhost}'" appBase="mxx'$i'"\n unpackWARs="true" autoDeploy="true">\n <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"\n prefix="mxx'$i'_access_log" suffix=".txt"\n pattern="%h %l %u %t "%r" %s %b %{JSESSIONID}c %{HOST}i %{X-Forwarded-For}i %{X-Via}o" />\n </Host>\n' \
${tomcat_path}/conf/server.xml && color "虛擬主機配置成功" 0
mkdir ${tomcat_path}/mxx${i}/ROOT -p
chown tomcat.tomcat ${tomcat_path}/mxx$i -R
cat > ${tomcat_path}/mxx$i/ROOT/index.jsp <<\EOF
<%@ page import="java.util.*" %>
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" http-equiv=Content-Type content="text/html;charset=utf-8">
<title>tomcat test</title>
</head>
<body>
<h1> Tomcat Website </h1>
<div>On <%=request.getServerName() %></div>
<div><%=request.getLocalAddr() + ":" + request.getLocalPort() %></div>
<div>SessionID = <span style="color:blue"><%=session.getId() %></span></div>
<%=new Date()%>
<h1>這是一個測試頁面</h1>
</body>
</html>
EOF
systemctl restart tomcat.service
done
}
#exec
jdk_install && tomcat_install && tomcat_config
測試結果
image.png
image.png
4村视、nginx實現(xiàn)后端tomcat的負載均衡調度
Nginx作為反向代理配置
配置nginx反向代理官套，將域名全部調度給后端tomcat服務器組，因為后端tomcat配置了虛擬主機，因此調度時修改了Host字段的值：
[root@centos8mini ~]# cat /data/nginx/conf/conf.d/server1.conf
server {
listen 80;
server_name m1.mxx.com;
default_type text/html;
root /data/server1;
# location ~* \.jsp$ {
# proxy_pass http://www.mxx.com:8080;
# #proxy_set_header Host $http_host;
# }
# location ~* \.html$ {
# proxy_pass http://blog.mxx.com:8080;
# #proxy_set_header Host $http_host;
# }
# location /blog {
# index index.jsp;
# proxy_pass http://www.mxx.com:8080/blog;
# }
location / {
index index.jsp;
proxy_set_header Host www.mxx.com;
proxy_pass http://webserver;
}
}
在http語句塊下定義服務器組奶赔，指向后端的兩臺tomcat服務器惋嚎，調度算法也是在這里指定：
[root@centos8mini ~]# cat /data/nginx/conf/nginx.conf
...
http {
upstream webserver {
#hash $cookie_jsessionid consistent;
server 192.168.1.12:8080 max_fails=3 fail_timeout=30s;
server 192.168.1.14:8080 max_fails=3 fail_timeout=30s;
....
}
...
測試結果
修改windows的hosts文件，解析m1.mxx.com域名站刑，指向nginx代理服務器：
192.168.1.13 m1.mxx.com m2.mxx.com
沒做session的會話保持前另伍，調度到不同服務器會生成不同的session ID：
image.png
5、簡述memcached的工作原理
Memcached采用Slab Allocator機制來分配绞旅、管理內存摆尝。根據提前定義好的Growth factor等參數(shù)，memcached啟動后自動按照定義的參數(shù)分配內存空間并執(zhí)行數(shù)據的存儲玻靡。
memcached使用c/s架構结榄，服務器維護key-value關聯(lián)數(shù)組，客戶端輸入數(shù)據到數(shù)組中囤捻，并使用key來查詢臼朗。
key可以有250字節(jié)長度，value可以存最大1M蝎土。
memcached內存空間是啟動時提前分配好的视哑。內存被分成大小相等的slab，然后每個slab被等分成chunk誊涯，slab按照chunk的大小進行分組挡毅，組成不同的slab class。
- 一個1M大小內存空間作為一個slab暴构，slab會等分成固定大小的chunk跪呈，但不同slab等分的chunk是不同的。處于相同層次的slab分配的chunk大小是相同的取逾，不同層次按照growth factor來確定下一層次slab中chunk的大小耗绿。
當執(zhí)行value存儲時，memcached會根據value的大小砾隅，找到合適的chunk進行存儲误阻。
當內存不足時，memcached使用LRU（least recently used）機制來查找可用空間晴埂。memcached不會監(jiān)視數(shù)據是否過期究反，而是采取Lazy Expiration懶過期機制，在取數(shù)據時才查看數(shù)據是否過期儒洛，過期就把數(shù)據有效期標識為0精耐，但并不清理該數(shù)據，而是在以后直接覆蓋該位置來存儲其他數(shù)據琅锻。
memcached集群功能是基于客戶端的分布式集群黍氮，客戶端在生成數(shù)據時決定應該在哪里存放數(shù)據唐含，memcached不會去執(zhí)行數(shù)據的集群同步。
