概述
?對(duì)于一個(gè)ceph開發(fā)人員來(lái)說(shuō)編譯源碼以及打rpm是其必備技能。無(wú)論是fix bug還是向社區(qū)提交pull request都離不開編譯源碼。而rpm包又是linux系統(tǒng)發(fā)行產(chǎn)品時(shí)常用的格式。在官網(wǎng)和其他blog可以很多這方面的介紹宜狐，我在這里把自己編譯的步奏，以及遇到的坑及解決辦法記錄一下，僅供新人參考踪央。
編譯源碼
環(huán)境介紹：
- ceph version: Luminous 12.2.10
- 硬件環(huán)境：Centos7虛擬機(jī)/Centos7 docker container
準(zhǔn)備工作
- 安裝yum源
這些源用于安裝常用工具，執(zhí)行install-reps.sh安裝以來(lái)包使用瓢阴，列表如下：
-rw-r--r--. 1 root root 2523 Jan 8 07:10 CentOS-Base.repo
-rw-r--r--. 1 root root 1309 Jan 8 08:22 CentOS-CR.repo
-rw-r--r--. 1 root root 649 Nov 23 13:16 CentOS-Debuginfo.repo
-rw-r--r--. 1 root root 314 Nov 23 13:16 CentOS-fasttrack.repo
-rw-r--r--. 1 root root 630 Nov 23 13:16 CentOS-Media.repo
-rw-r--r--. 1 root root 916 Jan 8 07:10 CentOS-SCLo-scl.repo
-rw-r--r--. 1 root root 898 Jan 8 07:10 CentOS-SCLo-scl-rh.repo
-rw-r--r--. 1 root root 1331 Nov 23 13:16 CentOS-Sources.repo
-rw-r--r--. 1 root root 5701 Nov 23 13:16 CentOS-Vault.repo
-rw-r--r--. 1 root root 664 Jan 8 07:10 epel.repo
-rw-r--r--. 1 root root 951 Oct 2 2017 epel.repo.rpmnew
-rw-r--r--. 1 root root 1050 Oct 2 2017 epel-testing.repo
-rw-r--r--. 1 root root 155 Jan 8 07:10 mirrors.aliyun.com_epel_7_x86_64_.repo
-rw-r--r--. 1 root root 153 Jan 8 07:10 mirrors.aliyun.com_epel_x86_64_.repo
- 安裝git
yum install git
You have enabled checking of packages via GPG keys. This is a good thing.
However, you do not have any GPG public keys installed. You need to download
the keys for packages you wish to install and install them.
You can do that by running the command:
rpm --import public.gpg.key
從別的機(jī)器拷貝/etc/ceph/rpm-gpg 過(guò)來(lái),然后執(zhí)行
rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
下載12.2.10源碼
git clone -b v12.2.10 --single-branch git://github.com/ceph/ceph.git master更新子模塊代碼
git submodule update —init —recursive
下面兩步如果環(huán)境已經(jīng)有了切版本合適就可以跳過(guò)
- 安裝gcc
git clone [https://github.com/gcc-mirror/gcc.git](https://github.com/gcc-mirror/gcc.git)
b.mkdir build ; cd build
../configure --prefix=/usr --disable-multilib
c.yum install -y gmp-devel libmpc-devel mpfr-devel flex
d.make -j16 > log 2>&1 && make install >log 2>&1 &
- 升級(jí)cmake
wget https://cmake.org/files/v3.9/cmake-3.9.2.tar.gz
//解壓后自行安裝
編譯
- 執(zhí)行install-deps.sh來(lái)安裝編譯時(shí)需要的各種依賴包
sh install-deps.sh
Note:
1.這一步一般要耗費(fèi)較長(zhǎng)時(shí)間畅蹂，所以我們可以執(zhí)行這一步的時(shí)候打開yum的緩存模式，把依賴包緩存在本地荣恐，隨后把這些rpm包建立一個(gè)自己的yum源液斜，這樣下次安裝的時(shí)候就從本地的yum源安裝，會(huì)大大縮減時(shí)間!
[root@xt-53 ceph-12.2.10]# cat /etc/yum.conf
[main]
cachedir=/var/cache/yum/$basearch/$releasever
keepcache=1 //置為1叠穆，打開緩存模式
debuglevel=2
...
緩存的包會(huì)保留在/var/cache/yum/x86_64/7/ 下面
2.如果是容器內(nèi)編譯也可以把安裝完依賴包的容器提交到新的鏡像少漆，之后用這個(gè)鏡像啟動(dòng)的容器就不用再安裝依賴包了。
3.install-deps.sh帶的默認(rèn)的yum源比較慢硼被，可以更改更快的yum源
install-deps.sh中
$SUDO yum-config-manager --add-repo https://dl.fedoraproject.org/pub/epel/$VERSION_ID/x86_64/
更改為
$SUDO yum-config-manager --add-repo https://mirrors.aliyun.com/epel/$VERSION_ID/x86_64/
執(zhí)行do_cmake.sh
sh do_cmake.sh開始make
make -j16 > log 2>&1 & //線程數(shù)等于cpu core的2倍示损，可以提高編譯的速度
當(dāng)然我們也可以只編譯某一個(gè)模塊,比如只編譯client就可以
make client > log 2>&1 &
最小測(cè)試用例
?當(dāng)我們更改了代碼準(zhǔn)備提交到公司內(nèi)部repo或者社區(qū)repo都需要先執(zhí)行一下最小測(cè)試集，看看自己修改的代碼有沒(méi)有影響到別的模塊(社區(qū)也會(huì)進(jìn)行同樣的測(cè)試)嚷硫。
- ctest
可以單獨(dú)跑一個(gè)測(cè)試用例检访，若不加參數(shù)就跑所有的測(cè)試，和make一樣仔掸，可以用-j選項(xiàng)來(lái)多線程同時(shí)跑脆贵。
例如：
# ctest -R cephfs
Test project /home/ivan/ws/Luminous/code/ceph-12.2.10/build
Start 42: unittest_libcephfs_config
1/1 Test #42: unittest_libcephfs_config ........ Passed 0.02 sec
100% tests passed, 0 tests failed out of 1
Total Test time (real) = 0.06 sec
?也可以用-V選項(xiàng)來(lái)使運(yùn)行過(guò)程打印到前臺(tái)
# ctest -V -R client
UpdateCTestConfiguration from :/home/ivan/ws/Luminous/code/ceph-12.2.10/build/DartConfiguration.tcl
UpdateCTestConfiguration from :/home/ivan/ws/Luminous/code/ceph-12.2.10/build/DartConfiguration.tcl
Test project /home/ivan/ws/Luminous/code/ceph-12.2.10/build
Constructing a list of tests
Done constructing a list of tests
Checking test dependency graph...
Checking test dependency graph end
test 138
Start 138: unittest_mclock_client_queue
138: Test command: /home/ivan/ws/Luminous/code/ceph-12.2.10/build/bin/unittest_mclock_client_queue
138: Test timeout computed to be: 3600
138: [==========] Running 4 tests from 1 test case.
138: [----------] Global test environment set-up.
138: [----------] 4 tests from MClockClientQueueTest
138: [ RUN ] MClockClientQueueTest.TestSize
138: [ OK ] MClockClientQueueTest.TestSize (0 ms)
138: [ RUN ] MClockClientQueueTest.TestEnqueue
138: [ OK ] MClockClientQueueTest.TestEnqueue (0 ms)
138: [ RUN ] MClockClientQueueTest.TestEnqueueStrict
138: [ OK ] MClockClientQueueTest.TestEnqueueStrict (0 ms)
138: [ RUN ] MClockClientQueueTest.TestRemoveByClass
138: [ OK ] MClockClientQueueTest.TestRemoveByClass (0 ms)
138: [----------] 4 tests from MClockClientQueueTest (1 ms total)
138:
138: [----------] Global test environment tear-down
138: [==========] 4 tests from 1 test case ran. (1 ms total)
138: [ PASSED ] 4 tests.
1/1 Test #138: unittest_mclock_client_queue ..... Passed 0.04 sec
The following tests passed:
unittest_mclock_client_queue
100% tests passed, 0 tests failed out of 1
Total Test time (real) = 0.08 sec
- make check
編譯以及順便跑所有的測(cè)試用例，可以加-j選項(xiàng)來(lái)啟動(dòng)多線程起暮。
構(gòu)建RPM包
?當(dāng)我們需要發(fā)布產(chǎn)品時(shí)就需要把各個(gè)模塊打成rpm包卖氨。我把官網(wǎng)上面的環(huán)節(jié)再加上自己的特殊需求寫了個(gè)腳本來(lái)實(shí)現(xiàn)一鍵打包功能。
?我們首先需要更改make-disk，該腳本主要用于生成ceph.spec和ceph-*.tar.bz2文件以供后面打rpm包使用筒捺。我們主要改變了rpm_version,rpm_release 持搜，是否下載boost庫(kù)等.
# cat make-dist
#!/bin/sh -e
DOWNLOAD_BOOST=$1
if [ ! -d .git ]; then
echo "no .git present. run this from the base dir of the git checkout."
exit 1
fi
##############################################################
#
# Version is very import, we add a Version.txt to record it
# There should use own Version.txt instead git tag
#
##############################################################
version=`cat Version.txt | grep VERSION |awk -F '=' '{print $2}'`
[ -z "$version" ] && version=`git describe --match 'v*' | sed 's/^v//'`
outfile="ceph-$version"
echo "version $version"
# update submodules
echo "updating submodules..."
force=$(if git submodule usage 2>&1 | grep --quiet 'update.*--force'; then echo --force ; fi)
if ! git submodule sync || ! git submodule update $force --init --recursive; then
echo "Error: could not initialize submodule projects"
echo " Network connectivity might be required."
exit 1
fi
download_boost() {
boost_version=$1
shift
boost_md5=$1
shift
boost_version_underscore=$(echo $boost_version | sed 's/\./_/g')
boost_fname=boost_${boost_version_underscore}.tar.bz2
set +e
if [ "${DOWNLOAD_BOOST}" == "yes" ];then
while true; do
url_base=$1
shift
if [ -z $url_base ]; then
echo "Error: failed to download boost."
exit
fi
url=$url_base/$boost_fname
wget -c --no-verbose -O $boost_fname $url
if [ $? != 0 -o ! -e $boost_fname ]; then
echo "Download of $url failed"
elif [ $(md5sum $boost_fname | awk '{print $1}') != $boost_md5 ]; then
echo "Error: failed to download boost: MD5 mismatch."
else
break
fi
done
fi
set -e
tar xjf $boost_fname -C src \
--exclude="$boost_version_underscore/libs/*/doc" \
--exclude="$boost_version_underscore/libs/*/example" \
--exclude="$boost_version_underscore/libs/*/examples" \
--exclude="$boost_version_underscore/libs/*/meta" \
--exclude="$boost_version_underscore/libs/*/test" \
--exclude="$boost_version_underscore/tools/boostbook" \
--exclude="$boost_version_underscore/tools/quickbook" \
--exclude="$boost_version_underscore/tools/auto_index" \
--exclude='doc' --exclude='more' --exclude='status'
mv src/boost_${boost_version_underscore} src/boost
tar cf ${outfile}.boost.tar ${outfile}/src/boost
rm -rf src/boost
}
# clean out old cruft...
echo "cleanup..."
rm -f $outfile*
# build new tarball
echo "building tarball..."
bin/git-archive-all.sh --prefix ceph-$version/ \
--verbose \
--ignore corpus \
$outfile.tar
# populate files with version strings
echo "including src/.git_version, ceph.spec"
# The command finds the most recent tag that is reachable from a commit. If the tag points to the commit,
# then only the tag is shown. Otherwise, it suffixes the tag name with the number of additional commits on
# top of the tagged object and the abbreviated object name of the most recent commit. The result is a
# human-readable" object name which can also be used to identify the commit to other git commands.
(git rev-parse HEAD ; git describe) 2> /dev/null > src/.git_version
# if the version has '-' in it, it has a 'release' part,
# like vX.Y.Z-N-g<shortsha1>. If it doesn't, it's just
# vX.Y.Z. Handle both, and translate - to . for rpm
# naming rules (the - separates version and release).
if expr index $version '-' > /dev/null; then
rpm_version=`echo $version | cut -d - -f 1-1`
rpm_release=`echo $version | cut -d - -f 2- | sed 's/-/./'`
else
rpm_version=$version
# rpm style: ceph-"module"-"version"-"commit_id"
# example: ceph-mds-12.2.10.mh.0-1.ga200519.el7.x86_64.rpm
rpm_release=`tail -n 1 src/.git_version | cut -d - -f 2- | sed 's/-/./'`
fi
for spec in ceph.spec.in alpine/APKBUILD.in; do
cat $spec |
sed "s/@VERSION@/$rpm_version/g" |
sed "s/@RPM_RELEASE@/$rpm_release/g" |
sed "s/@TARBALL_BASENAME@/ceph-$version/g" > `echo $spec | sed 's/.in$//'`
done
ln -s . $outfile
tar cvf $outfile.version.tar $outfile/src/.git_version $outfile/ceph.spec $outfile/alpine/APKBUILD
# NOTE: If you change this version number make sure the package is available
# at the three URLs referenced below (may involve uploading to download.ceph.com)
boost_version=1.66.0
download_boost $boost_version b2dfbd6c717be4a7bb2d88018eaccf75 \
https://dl.bintray.com/boostorg/release/$boost_version/source \
https://downloads.sourceforge.net/project/boost/boost/$boost_version \
https://download.ceph.com/qa
tar --concatenate -f $outfile.all.tar $outfile.version.tar
tar --concatenate -f $outfile.all.tar $outfile.boost.tar
tar --concatenate -f $outfile.all.tar $outfile.tar
mv $outfile.all.tar $outfile.tar
rm $outfile
rm -f $outfile.version.tar
rm -f $outfile.boost.tar
echo "compressing..."
bzip2 -9 $outfile.tar
echo "done."
腳本make-rpm.sh如下：
# cat make-rpm.sh
#!/bin/bash
#echo $@
#######################################################
#
# build Ceph rpm packages
#
# ./make-rpm.sh
#
########################################################
CLEANBUILD="yes"
INSTALL_DEPS="false"
COMMAND=$1
DOWNLOAD_BOOST="false"
# current dir of this script
BASEDIR=`cd $(dirname $0); pwd -P`
function usage()
{
cat <<EOF
Usage: ./$0 <build|clean> [Flags]
Build ceph and generate rpm packages.
Flags:
-i,--install-deps //don't excute install-deps.sh
-n,--no-cleanbuild //don't clean build
--download_boost // download boost library
-h,--help //list help info
EOF
}
function build_rpms()
{
# we don't need install deps every time
if [ "$INSTALL_DEPS" == "yes" ];then
echo "sh ./install-deps.sh"
#sh ./install-deps.sh
fi
if [ "$MODULE" != "" ];then
echo "compile $MODULE"
else
echo "compile all modules"
fi
# we only need download boot at first time
if [ "$DOWNLOAD_BOOST" == "yes" ];then
echo "make-dist yes"
./make-dist yes
else
echo "make-dist false"
./make-dist false
fi
###########################################
#
# Version is very import, we add a Version.txt to record it
#
###########################################
version=`cat Version.txt | grep VERSION |awk -F '=' '{print $2}'`
rm -rf ${BASEDIR}/rpmbuild
mkdir ${BASEDIR}/rpmbuild && cd ${BASEDIR}/rpmbuild && mkdir BUILD BUILDROOT RPMS SOURCES SPECS SRPMS
cp ${BASEDIR}/ceph-${version}.tar.bz2 ${BASEDIR}/rpmbuild/SOURCES
tar --strip-components=1 -C ${BASEDIR}/rpmbuild/SPECS/ --no-anchored -xvjf ${BASEDIR}/rpmbuild/SOURCES/ceph-${version}.tar.bz2 "ceph.spec"
if [ "$CLEANBUILD" == "yes" ];then
#rpmbuild --define="_topdir "${BASEDIR}"" -ba ${BASEDIR}/rpmbuild/SPECS/ceph.spec --with clean_build
rpmbuild --define="_topdir "${BASEDIR}"/rpmbuild" -ba ./SPECS/ceph.spec --with clean_build
else
#rpmbuild --define '_topdir ./rpmbuild' -ba ${BASEDIR}/rpmbuild/SPECS/ceph.spec
rpmbuild --define="_topdir "${BASEDIR}"/rpmbuild" -ba ./SPECS/ceph.spec
fi
mv ${BASEDIR}/rpmbuild/RPMS/*/* ${BASEDIR}/rpmbuild/
mv ${BASEDIR}rpmbuild/SRPMS/* ${BASEDIR}/rpmbuild/
}
function make_clean()
{
echo "clean last build result ..."
rm -rf rpmbuild
}
function do_work()
{
if [ "$COMMAND" == "build" ];then
echo "start building ..."
build_rpms
else
echo "start cleaning"
make_clean
fi
}
ARGS=`getopt -o inmc --long install-deps,no-cleanbuild,module:,download_boost,clean -n 'example.sh' -- "$@"`
eval set -- "${ARGS}"
while true
do
case "$1" in
-i|--install-deps)
INSTALL_DEPS="yes"
echo INSTALL_DEPS $INSTALL_DEPS
shift
;;
-n|--no-cleanbuild)
CLEANBUILD="false";
echo CLEANBULD: $CLEANBUILD;
shift
;;
--download_boost)
DOWNLOAD_BOOST="yes";
echo DOWNLOAD_BOOST $DOWNLOAD_BOOST;
shift
;;
-c|--clean)
COMMAND="clean";
echo COMMAND: $COMMAND;
shift
;;
--)
shift;
break;;
*) echo "Internal error!;"; exit 1 ;;
esac
done
if [ $# -lt 1 ];then
usage
exit -1
fi
do_work
- 腳本使用方法
# sh make-rpm.sh -h
Usage: ./make-rpm.sh <build|clean> [Flags]
Build ceph and generate rpm packages.
Flags:
-i,--install-deps //don't excute install-deps.sh
-n,--no-cleanbuild //don't clean build
--download_boost // download boost library
-h,--help //list help info
End.
- build rpm 包
// 開始打包,不安裝依賴包，且不下載boost庫(kù)
sh make-rpm.sh build
- 清除掉上次編譯的殘留文件
sh make-rpm.sh clean
或者
sh make-rpm.sh -c
