持續(xù)集成CI
[Chapter 零] * Ref.
基于內(nèi)部研發(fā)框架pactera jeesit版本 1.2.11-SNAPSHOT 發(fā)行之際弱判，引入CI服務(wù)Jenkins膝藕，并通過In-house Conf.內(nèi)部大會分享蔓涧，特此整理成文以資備案和參考较剃。 By Jeffen Apr. 15 , 2016
[Chapter Ⅰ] * Continuous Integration
- 最核心的敏捷（Agile）實(shí)踐辩蛋，--持續(xù)集成（Continuous Integration），
- 可以快速翩蘸、高頻率、自動淮逊、構(gòu)建催首、所有源碼，并反饋泄鹏。
- 在不斷變化的需求中快速適應(yīng)和保證軟件的質(zhì)量郎任。
- 倡導(dǎo)團(tuán)隊開發(fā)成員必須經(jīng)常集成他們的工作，甚至每天都可能發(fā)生多次集成备籽。
- 而每次的集成都是通過自動化的構(gòu)建來驗(yàn)證涝滴，包括自動編譯、發(fā)布和測試胶台，從而盡快地發(fā)現(xiàn)集成錯誤歼疮，
- 讓團(tuán)隊能夠更快的開發(fā)內(nèi)聚的軟件。
持續(xù)集成的核心價值在于：
- 持續(xù)集成中的任何一個環(huán)節(jié)都是自動完成的诈唬，無需太多的人工干預(yù)韩脏，有利于減少重復(fù)過程以節(jié)省時間、費(fèi)用和工作量铸磅；
- 持續(xù)集成保障了每個時間點(diǎn)上團(tuán)隊成員提交的代碼是能成功集成的赡矢。換言之杭朱，任何時間點(diǎn)都能第一時間發(fā)現(xiàn)軟件的集成問題，使任意時間發(fā)布可部署的軟件成為了可能吹散；
- 持續(xù)集成還能利于軟件本身的發(fā)展趨勢弧械，這點(diǎn)在需求不明確或是頻繁性變更的情景中尤其重要，持續(xù)集成的質(zhì)量能幫助團(tuán)隊進(jìn)行有效決策空民，同時建立團(tuán)隊對開發(fā)產(chǎn)品的信心刃唐。
持續(xù)集成的原則：
- 需要版本控制軟件保障團(tuán)隊成員提交的代碼不會導(dǎo)致集成失敗。常用的版本控制軟件有 IBM Rational ClearCase界轩、CVS画饥、Subversion 等；
- 開發(fā)人員必須及時向版本控制庫中提交代碼浊猾，也必須經(jīng)常性地從版本控制庫中更新代碼到本地抖甘；
- 需要有專門的集成服務(wù)器來執(zhí)行集成構(gòu)建。根據(jù)項(xiàng)目的具體實(shí)際葫慎，集成構(gòu)建可以被軟件的修改來直接觸發(fā)衔彻，也可以定時啟動，如每半個小時構(gòu)建一次偷办；
7） 必須保證構(gòu)建的成功米奸。如果構(gòu)建失敗，修復(fù)構(gòu)建過程中的錯誤是優(yōu)先級最高的工作爽篷。一旦修復(fù)悴晰，需要手動啟動一次構(gòu)建。
引入Hudson3.3.3持續(xù)集成發(fā)布定制的框架版本逐工，過程：
- 持續(xù)編譯铡溪，SVN
- 持續(xù)數(shù)據(jù)庫腳本過程集成
- 持續(xù)測試，Junit
- 持續(xù)審查泪喊，引入Checkstyle 棕硫、PMD持續(xù)審查代碼BadSmell；引入FindBug持續(xù)進(jìn)行代碼auto review袒啼；哈扮，產(chǎn)出各類check report；
- 持續(xù)部署蚓再，容器環(huán)境相關(guān)滑肉，Cargo
- 持續(xù)反饋，將集成失敗報告發(fā)給本次集成的相關(guān)代碼提交者摘仅，CC項(xiàng)目經(jīng)理靶庙。
因svn plugin地址中文尋址問題及Ugly UI，放棄hudson娃属，
引入 Jenkins1.656 持續(xù)集成六荒，前身是hudson护姆，做了擴(kuò)展和包裝，UI/UX交互友好掏击，apache foundation開源項(xiàng)目卵皂。
[Chapter Ⅱ] * Install
2.1 (×)Hudson
- Hudson downloads
- Installing Hudson
- Hudson 3.3.2 (ChangeLog)
- Installing Hudson as a Windows service
2.2 (√)Jenkins
- https://jenkins.io/index.html
- √ recommend, run as windows server by background
- https://jenkins.io/content/thank-you-downloading-windows-installer#stable
- run as terminal
- http://www.ibm.com/developerworks/cn/java/j-lo-jenkins/index.html
2.3 Plugins
2.3.1 FindBugs
2.3.1.1 Description
- 尋找代碼缺陷：
- 未關(guān)閉的數(shù)據(jù)庫連接；
- 缺少必要的null check;
- 多余的null check;
- 多余的if后置條件砚亭；
- 相同的條件分支灯变；
- 重復(fù)的代碼塊；
- 錯誤的使用了“＝＝”钠惩；
- 建議使用stringbuffer代替字符串連加柒凉；
2.3.1.2 URL
- http://wiki.hudson-ci.org/display/HUDSON/FindBugs+Plugin
- https://wiki.jenkins-ci.org/display/JENKINS/FindBugs+Plugin
2.3.1.3 Commands
- mvn findbugs:help 查看findbugs插件的幫助
- mvn findbugs:check 檢查代碼是否通過findbugs檢查族阅，如果沒有通過檢查篓跛，檢查會失敗，但檢查不會生成結(jié)果報表
- mvn findbugs:findbugs 檢查代碼是否通過findbugs檢查坦刀，如果沒有通過檢查愧沟，檢查不會失敗，會生成結(jié)果報表保存在target/findbugsXml.xml文件中
- mvn findbugs:gui 檢查代碼并啟動gui界面來查看結(jié)果
2.3.2 PMD
2.3.3 CheckStyle
[Chapter Ⅲ] * Run
3.1 Enviroments
JDK
- %JAVA_HOME% : c:\Program Files\Java\jdk.1.7.0_67
Maven
- %M3_HOME%: e:maven/maven3
- E:\maven\maven3\conf\settings.xml
- 建議repository設(shè)定到nexus私服下
- 設(shè)定nexus的服務(wù)認(rèn)證以便deploy
<localRepository>D:/sonatype-work/nexus/storage/central</localRepository>
<!-- Nexus servers-->
<server>
<id>releases</id>
<username>admin</username>
<password>admin123</password>
</server>
<server>
<id>snapshots</id>
<username>admin</username>
<password>admin123</password>
</server>
- 如maven無法自動下載鲤遥，可手動安裝ojdbc14
cd E:\maven\repository
e:
mvn install:install-file -Dpackaging=jar -DgroupId=com.oracle -Dversion=10.2.0.4.0 -DartifactId=ojdbc14 -Dfile=ojdbc14-10.2.0.4.0.jar
3.2 (×)Hudson
Path: 192.168.5.5$d:/Hudson/
$ java –jar Hudson-3.3.2.war --httpPort=8082
3.3 (√)Jenkins
Path: 192.168.5.5$d:/jenkins/
$ java -Dhudson.util.ProcessTree.disable=true -jar jenkins.war --httpPort=8082
ProcessTree:
Jenkins ProcessTreeKiller. A more detailed explanation here. It's a design decision to kill any processes that are spawned by the build to maintain a clean environment. Unfortunately that means you can't leave a process (such as Tomcat) running after the build.
You can disable this functionality globally (not recommended) by launching Jenkins
Show:
3.4 URL
Jenkins
http://localhost:8082/
Install plugins on homepage by guidlines.
pactera jeesite
http://192.168.5.5:8080/pactera-jeesite
[Chapter Ⅳ] * Enjoy
4.1 Setting
http://localhost:8082/configure
System Configration>>
- JDK : jdk-1.7 (%JAVA_HOME%)
- Maven : maven-3.1.1 (%M3_HOME%)
- MAVEN_OPTS: -Xms128m –Xmx512m
4.2 Create
4.2.1 Job Configurations
- Build: a free-style software project
- Name: pactera-jeesite-aggregator
- Description：pactera jeesite 1.2.10+ web show
- Max # of builds to keep: 20 (only up to this number of build records are kept)
BUILDS:
- pactera-jeesite-common
- pactera-jeesite-cms
- pactera-jeesite-gen
- pactera-jeesite-oa
- pactera-jeesite-oa-core
- pactera-jeesite-web
- pactera-jeesite-web-dist
4.2.2 Source Code Management
Subversion svn url :
http://192.168.5.5:8088/svn/產(chǎn)品/jeesite/trunk/aggregator
Repository URL:
http://192.168.5.5:8088/svn/產(chǎn)品/jeesite/trunk/aggregator
Click “enter credential / update credential” to subversion authentication.
URLS:
- http://192.168.5.5:8088/svn/產(chǎn)品/jeesite/trunk/jeesite-common
- http://192.168.5.5:8088/svn/產(chǎn)品/jeesite/trunk/jeesite-cms
- http://192.168.5.5:8088/svn/產(chǎn)品/jeesite/trunk/jeesite-gen
- http://192.168.5.5:8088/svn/產(chǎn)品/jeesite/trunk/jeesite-oa
- http://192.168.5.5:8088/svn/產(chǎn)品/jeesite/trunk/jeesite-oa-core
- http://192.168.5.5:8088/svn/產(chǎn)品/jeesite/trunk/jeesite-soa
- http://192.168.5.5:8088/svn/產(chǎn)品/jeesite/trunk/jeesite-web
- http://192.168.5.5:8088/svn/產(chǎn)品/jeesite/trunk/jeesite-web-dist
4.2.3 Build Triggers
√ Poll SCM
- 第一個參數(shù)代表的是分鐘 minute沐寺，取值 0~59；
- 第二個參數(shù)代表的是小時 hour盖奈，取值 0~23混坞；
- 第三個參數(shù)代表的是天 day，取值 1~31钢坦；
- 第四個參數(shù)代表的是月 month究孕，取值 1~12；
- 最后一個參數(shù)代表的是星期 week爹凹，取值 0~7厨诸，0 和 7 都是表示星期天。
說明：構(gòu)建前提是禾酱，配置的代碼庫有更新微酬，再觸發(fā)。
Schedule :*/59 * * * *(每一個小時輪尋源碼倉庫颤陶，發(fā)現(xiàn)更新時構(gòu)建本項(xiàng)目)
Schedule :00 01 * * 1-5(每周一到周五凌晨一點(diǎn))
4.2.4 Build
4.2.4.1 Invoke top-level maven targets
- Maven Version: maven-3.1.1
-
Golals(-aggrerator/-web-dist):
clean deploy -Pdev_pdt -Dmaven.test.skip=true -
Golals:
clean deploy -Dmaven.test.skip=true
(×)Hudson
(√)Jenkins
4.2.4.2 (×)Execute Windows batch command type1
For project: pactera-jeesite-web-dist
cd D:\apache-tomcat-6.0.20-jeecg\bin
d:
restart.bat
Restart tomcat by %CATALINA_HOME%/bin/restart.bat for Windows:
@echo restart tomcat6 service
@echo Created by Jeffen@pactera
@echo Date 2016/4/7
@echo Version 0.1
@echo 1. stop tomcat6 service
@echo 2. sleep 10 seconds
@echo 3. start tomcat6 service
echo --------------------------------------------------------
echo [%date%%time%] prepare to restart tomcat
echo [%date%%time%] stop tomcat
call "shutdown.bat"
@echo waiting 10 seconds
ping 127.0.0.1 -n 10
echo [%date%%time%] restart tomcat
call "startup.bat"
echo --------------------------------------------------------
echo
4.2.4.3 (√)Execute Windows batch command type2
http://stackoverflow.com/questions/6204003/kill-a-process-by-looking-up-the-port-being-used-by-it-from-a-bat
a. 先殺進(jìn)程颗管，以防deploy時無法刪除target folder：
cd D:\apache-tomcat-6.0.20-jeecg\bin
d:
call "kill_process_by_port.bat"
b. 再maven deploy發(fā)布項(xiàng)目：
clean deploy -Pdev_pdt -Dmaven.test.skip=true
c. 最后重啟tomcat container：
cd D:\apache-tomcat-6.0.20-jeecg\bin
d:
@echo [%date%%time%] restart tomcat
call "startup.bat"
@echo waiting 30 seconds
ping 127.0.0.1 -n 30
Kill tomcat by %CATALINA_HOME%/bin/kill_process_by_port.bat for Windows:
@echo kill tomcat service.
@echo if run by terminal then change %%P to %P
@echo Created by Jeffen@pactera
@echo Date 2016/4/14
@echo Version 0.1
@echo [%date%%time%] kill tomcat container process by port
FOR /F "tokens=5 delims= " %%P IN ('netstat -a -n -o ^| findstr :8080.*LISTENING') DO TaskKill.exe /F /PID %%P
@echo waiting 5 seconds
ping 127.0.0.1 -n 5
HardCopy:
4.2.4.4 Publish FindBugs Analysis results
4.2.4.5 Publish PMD Analysis results
4.2.5 Reports
4.2.5.1 FindBugs Warinings
Solution：
There are different ways of encoding a String as bytes -- the charset determines that encoding. If you don't specify a charset, as in your call to str.getBytes(), it uses the system default.
FindBugs is warning you about this because you should think about what encoding you want to use for your output. If you're writing to a file, what are the readers of that file expecting? It is safest if you can specify an explicit encoding for the file so you don't write it one way and read it another way.
To specify an explicit charset, use str.getBytes(Charset.forName("UTF-8")), for example. UTF-8 is a good choice because it is always supported and can encode any character.
For example, .properties files are always ISO 8859-1 (i.e. Latin-1). That's documented so there is no ambiguity around what encoding to use.
[Chapter Ⅴ] * FAQ&ISSUE
5.1 Hudson Svn E2049000 could not open the requested SVN file system
ISSUE:
FAILED: svn: E204900: Could not open the requested SVN filesystem
org.tmatesoft.svn.core.SVNException: svn: E204900: Could not open the requested SVN filesystem
svn: E175002: OPTIONS of '/svn/%C3%AF%C2%BF%C2%BD%C3%AF%C2%BF%C2%BD%C3%86%C2%B7/jeesite/trunk/aggregator': 500 Internal Server Error (http://192.168.5.5:8088)
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:64)
Solution：
hudson subversion plugin 不支持中文路徑，更換jenkins滓走。
因后者源于hudson 忙上，之前的配置大部分可自動復(fù)用！
jenkins對hudson某些插件做了調(diào)整闲坎，對界面UX設(shè)計做了極大的改進(jìn)疫粥。
5.2 Build Failure ：download ojdbc14
ISSUE:
[ERROR] Failed to execute goal on project jeesite-common: Could not resolve dependencies for project com.pactera.jeesite:jeesite-common:jar:1.2.10: Could not transfer artifact com.oracle:ojdbc14:jar:10.2.0.4.0 from/to oschina-repos (http://maven.oschina.net/content/groups/public): Connection to http://maven.oschina.net refused: Connection refused: connect
Solution：手動安裝ojdbc14茬斧，參看3.1章節(jié)
5.3 Batch NG：此時不應(yīng)有…
ISSUE:
FOR /F "tokens=5 delims= " %P IN ('netstat -a -n -o ^| findstr :8080.*LISTENING') DO TaskKill.exe /F /PID %P
Run by batch file, loging:此時不應(yīng)有8080*LISTENING
Solution：
批處理文件變量用 %%P 兩個百分號；DOS終端執(zhí)行時用 %P 一個百分號梗逮。
