一連試了幾天用集群的R構(gòu)建EVIDENCE,結(jié)果并不能讓我滿意,基本上一天只能跑一個早像,再運(yùn)行第二個的時候就會報錯僻肖。嘗試到現(xiàn)在就只剩下馬鈴薯的還未成功了,暫時先把問題撂在這卢鹦,我要去做ppt臀脏,下周二輪到我匯報。
R運(yùn)行的腳本如下:
library(org.Hs.eg.db)
swiss_id <- read.delim('/vol3/agis/zhoushaoqun_group/wangyantao/GO/swiss_go.stu1',header = F)
colnames(swiss_id) <- c('gene_id','GO')
ev_id <- select(org.Hs.eg.db,keys = as.vector(swiss_id$GO),columns = c('EVIDENCE'),keytype = "GO")
library(dplyr)
swiss_goev <- left_join(swiss_id,ev_id[,1:2])
write.csv(swiss_goev,'/vol3/agis/zhoushaoqun_group/wangyantao/GO/swiss_goev_stu.csv',row.names = F,quote = F)
運(yùn)行結(jié)果:
今天生信大神過來視察工作冀自,我逮住機(jī)會問了一下揉稚,大神要求看一下我的原始文件大小,我仔細(xì)看了一下熬粗,這馬鈴薯的swiss_go.stu1確實(shí)要比之前幾個物種的文件都要大上不少窃植,于是乎,打開一個荐糜,整整將近30萬行巷怜,而其他的頂多也就15萬行”┦希看來找到報錯的原因了延塑,文件太大,集群都搞不定答渔。
又于是乎关带,一拍腦袋,我把這個拆成兩個來搞不就好了嘛沼撕!說干就干:
文件1:前半部分(150000行)
library(org.Hs.eg.db)
swiss_id <- read.delim('/vol3/agis/zhoushaoqun_group/wangyantao/GO/swiss_go.stu1',header = F)
colnames(swiss_id) <- c('gene_id','GO')
ev_id <- select(org.Hs.eg.db,keys = as.vector(swiss_id$GO),columns = c('EVIDENCE'),keytype = "GO")
library(dplyr)
swiss_goev <- left_join(swiss_id,ev_id[,1:2])
write.csv(swiss_goev,'/vol3/agis/zhoushaoqun_group/wangyantao/GO/swiss_goev_stu1.csv',row.names = F,quote = F)
后半部分(149999行)
library(org.Hs.eg.db)
swiss_id <- read.delim('/vol3/agis/zhoushaoqun_group/wangyantao/GO/swiss_go.stu2',header = F)
colnames(swiss_id) <- c('gene_id','GO')
ev_id <- select(org.Hs.eg.db,keys = as.vector(swiss_id$GO),columns = c('EVIDENCE'),keytype = "GO")
library(dplyr)
swiss_goev <- left_join(swiss_id,ev_id[,1:2])
write.csv(swiss_goev,'/vol3/agis/zhoushaoqun_group/wangyantao/GO/swiss_goev_stu2.csv',row.names = F,quote = F)
果然如我所料宋雏,兩個文件順利拿到,平均39個GB务豺。下面就是要把得到的文件進(jìn)行合并:
合并文件(提交任務(wù))
merge_stu.sh
cat /vol3/agis/zhoushaoqun_group/wangyantao/GO/swiss_goev_stu1.csv /vol3/agis/zhoushaoqun_group/wangyantao/GO/swiss_goev_stu2.csv > /vol3/agis/zhoushaoqun_group/wangyantao/GO/swiss_goev_stu.csv
qsub -l mem=10G,nodes=1:ppn=4 /vol3/agis/zhoushaoqun_group/wangyantao/GO/merge_stu.sh
得到的文件竟然達(dá)到了77個GB:
還不知道這貨后面能不能讀取磨总,不行的話可能還得接著拆。笼沥。蚪燕。娶牌。