發(fā)簡信
IP屬地：北京
-
-
Oracle | 用戶、權(quán)限、角色管理操作一、用戶管理 1、創(chuàng)建用戶 2、修改用戶 3盘寡、刪除用戶 二、權(quán)限管理 1撮慨、系統(tǒng)權(quán)限 - 授權(quán)操作 2竿痰、對(duì)象權(quán)限 - 授權(quán)操作 3、回收權(quán)限操作 4砌溺、查看用戶和權(quán)限 三影涉、角色管...
-
樓主的配置有問題，應(yīng)該把多個(gè)key相同的屬性配置在一起
val sc = new SparkContext()
// 多個(gè)HDFS的相同配置
sc.hadoopConfiguration.setStrings("fs.defaultFS", "hdfs://cluster1", "hdfs://cluster2");
sc.hadoopConfiguration.setStrings("dfs.nameservices", "cluster1", "cluster2");
// cluster1的配置
sc.hadoopConfiguration.set("dfs.ha.namenodes.cluster1", "nn1,nn2");
sc.hadoopConfiguration.set("dfs.namenode.rpc-address.cluster1.nn1", "namenode001:8020");
sc.hadoopConfiguration.set("dfs.namenode.rpc-address.cluster1.nn2", "namenode002:8020");
sc.hadoopConfiguration.set("dfs.client.failover.proxy.provider.cluster1", "org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider");
// cluster2的配置
sc.hadoopConfiguration.set("dfs.ha.namenodes.cluster2", "nn3,nn4");
sc.hadoopConfiguration.set("dfs.namenode.rpc-address.cluster2.nn3", "namenode003:8020");
sc.hadoopConfiguration.set("dfs.namenode.rpc-address.cluster2.nn4", "namenode004:8020");
sc.hadoopConfiguration.set("dfs.client.failover.proxy.provider.cluster2", "org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider");chinfun 評(píng)論自Spark操作多HDFS集群 -
-
讓Spark-Streaming在Yarn上長時(shí)間運(yùn)行對(duì)于長時(shí)間運(yùn)行的Spark Streaming作業(yè)，一旦提交到Y(jié)ARN群集便需要永久運(yùn)行猖闪，直到有意停止鲜棠。任何中斷都會(huì)引起嚴(yán)重的處理延遲，并可能導(dǎo)致數(shù)據(jù)丟失或重復(fù)培慌。YARN和A...
-
Spark讀寫HBase實(shí)踐
Spark經(jīng)常會(huì)讀寫一些外部數(shù)據(jù)源豁陆，常見的有HDFS、HBase吵护、JDBC盒音、Redis竖配、Kafka等。這些都是Spark的常見操作里逆，做一個(gè)簡單的Demo總結(jié)，方便后續(xù)開發(fā)查閱...
-
我是大豬、一個(gè)讓你沉迷于文章內(nèi)容的IT寫作者偎血！
-
專注于大數(shù)據(jù)平臺(tái)化颇玷、數(shù)據(jù)可觀測性和數(shù)據(jù)可視化
-
DataX 3.0簡介 安裝及使用DataX3.0離線同步工具介紹 一. DataX3.0概覽 ?DataX 是一個(gè)異構(gòu)數(shù)據(jù)源離線同步工具笨农，致力于實(shí)現(xiàn)包括關(guān)系型數(shù)據(jù)庫(MySQL、Oracle等)帖渠、HDFS谒亦、...
-
Spark 及其生態(tài)的一些優(yōu)化與實(shí)踐
-
個(gè)人網(wǎng)站 : http://www.xcan.top 個(gè)人店鋪: 鞋盒: https://shop117015562.taobao.com/ 男裝: https://shop469108515.taobao.com/ Hadoop,Hbase,Hive,Spark,Apache Kafka,Apache Storm,Map reduce,SQOOP,Zookeeper
-
深入解析Spark Core、Spark Streaming源碼哼审，徹底精通Spark
-
-
如果你是程序員旁赊，或者有一顆喜歡寫程序的心桦踊，喜歡分享技術(shù)干貨、項(xiàng)目經(jīng)驗(yàn)终畅、程序員日常囧事等等籍胯，歡迎投稿《程序員》專題。 專題主編：小彤花園 http://www.reibang.com/users/4a4eb4feee62/ 【程序員】專題拒稿指北：http://www.reibang.com/p/7c8b33b5f63b 投稿須知： 1.收錄相關(guān)技術(shù)文章离福，但不限于技術(shù)杖狼，也可以是項(xiàng)目經(jīng)驗(yàn)類的文章和程序員日常。 2.文章內(nèi)不得有任何推廣信息妖爷。包括但不限于公眾號(hào)蝶涩、微博理朋、微信，更不得在通過投稿后再修改文章加入推廣信息绿聘。你可以將這些推廣以及個(gè)人信息放在個(gè)人主頁介紹里嗽上。 3.字?jǐn)?shù)極少，段落混亂熄攘，大段代碼沒有放到代碼框的文章不收錄 4.不相關(guān)的內(nèi)容不收錄兽愤，請(qǐng)確認(rèn)文章內(nèi)容和本專題相關(guān) 目前針對(duì)優(yōu)質(zhì)作者及愛好者，開設(shè)了官方程序員微信群挪圾，希望加入請(qǐng)發(fā)簡信給主編小彤花園索取個(gè)人微信浅萧，主編會(huì)將你加入到【程序員】微信群，分享內(nèi)容哲思，收獲知識(shí)洼畅。添加微信時(shí)備注昵稱。
-
Hi，很高興在@IT·互聯(lián)網(wǎng)專題遇見你忆嗜。 收錄范圍：收錄IT&互聯(lián)網(wǎng)行業(yè)觀察與思考己儒，數(shù)碼產(chǎn)品極客體驗(yàn)分析等類型文章 投稿須知：http://www.reibang.com/p/4edc76104c27 專題主編：@若錦 http://www.reibang.com/u/e4c63b354a77 官方社群：可聯(lián)系添加 @若錦 加入專題官方社群交流、學(xué)習(xí)捆毫，添加時(shí)備注簡書ID闪湾。
-
大數(shù)據(jù)途样，收錄大數(shù)據(jù)相關(guān)技術(shù)的文章。
-
本人公眾號(hào) “華仔聊技術(shù)”<br><br>十年互聯(lián)網(wǎng)人，個(gè)人IP&公眾號(hào)博主凛驮，主攻中間件相關(guān)硬核技術(shù)<br><br>公眾號(hào)kafka系列文章:<br><a href="https://links.jianshu.com/go?to=https%3A%2F%2Fmp.weixin.qq.com%2Fmp%2Fappmsgalbum%3F__biz%3DMzg3MTcxMDgxNA%3D%3D%26action%3Dgetalbum%26album_id%3D2147575846151290880%26scene%3D173%26from_msgid%3D2247488854%26from_itemidx%3D1%26count%3D3%26nolastread%3D1%23wechat_redirect" target="_blank">https://mp.weixin.qq.com/mp/appmsgalbum?__biz=Mzg3MTcxMDgxNA==&action=getalbum&album_id=2147575846151290880&scene=173&from_msgid=2247488854&from_itemidx=1&count=3&nolastread=1#wechat_redirect</a> 歡迎關(guān)注<br><br><br>github:<a href="https://links.jianshu.com/go?to=https%3A%2F%2Fgithub.com%2Fmenghuan" target="_blank">https://github.com/menghuan</a><br>PHP7/Redis5/Nginx/Linux源碼分析匯總鏈接:<a href="https://links.jianshu.com/go?to=https%3A%2F%2Fgithub.com%2Fmenghuan%2Flnmp-code-analysis" target="_blank">https://github.com/menghuan/lnmp-code-analysis</a><br><br>最近在寫的一款go cron工具：<a href="https://links.jianshu.com/go?to=https%3A%2F%2Fgithub.com%2Fmenghuan%2FgoDistributedCron" target="_blank">https://github.com/menghuan/goDistributedCron</a> 基本完成 后續(xù)會(huì)添加微服務(wù)相關(guān)設(shè)計(jì) 歡迎star<br><a href="https://links.jianshu.com/go?to=https%3A%2F%2Fgithub.com%2Fmenghuan%2Fgo-distributed-cron-fronted" target="_blank">https://github.com/menghuan/go-distributed-cron-fronted</a> go-cron vue管理后臺(tái) 待完善 歡迎star<br><br>友情鏈接<br>歐神的博客 <a href="https://links.jianshu.com/go?to=https%3A%2F%2Fblog.changkun.de%2F" target="_blank">https://blog.changkun.de/</a><br>碼農(nóng)桃花源 <a href="https://links.jianshu.com/go?to=https%3A%2F%2Fqcrao.com%2F" target="_blank">https://qcrao.com/</a><br>曹大的博客 <a href="https://links.jianshu.com/go?to=http%3A%2F%2Fxargin.com%2F" target="_blank">http://xargin.com/</a><br>面向信仰編程 <a href="https://links.jianshu.com/go?to=https%3A%2F%2Fdraveness.me" target="_blank">https://draveness.me</a><br>煎魚 <a href="https://links.jianshu.com/go?to=https%3A%2F%2Feddycjy.com%2Fposts%2F" target="_blank">https://eddycjy.com/posts/</a><br>達(dá)神的博客 <a href="https://links.jianshu.com/go?to=http%3A%2F%2Fwww.weisd.in%2F" target="_blank">http://www.weisd.in/</a><br>陳捷的博客 <a href="https://links.jianshu.com/go?to=http%3A%2F%2Fwww.chenjie.info%2F" target="_blank">http://www.chenjie.info/</a><br>夢康的博客 <a href="https://links.jianshu.com/go?to=https%3A%2F%2Fmengkang.net%2F" target="_blank">https://mengkang.net/</a><br>濤哥博客 <a href="https://links.jianshu.com/go?to=http%3A%2F%2Fwww.tiyee.net%2F" target="_blank">http://www.tiyee.net/</a><br>周至博客 <a href="https://links.jianshu.com/go?to=http%3A%2F%2Fwww.sunnysmilez.cn%2F" target="_blank">http://www.sunnysmilez.cn/</a><br>朱小廝博客 <a href="https://links.jianshu.com/go?to=https%3A%2F%2Fblog.csdn.net%2Fu013256816%2Farticle%2Fdetails%2F54743481" target="_blank">https://blog.csdn.net/u013256816/article/details/54743481</a><br>鯉魚的博客:<a href="https://links.jianshu.com/go?to=https%3A%2F%2Fwww.iamle.com" target="_blank">https://www.iamle.com</a><br>數(shù)據(jù)庫內(nèi)核月報(bào)<a href="https://links.jianshu.com/go?to=http%3A%2F%2Fmysql.taobao.org%2Fmonthly%2F" target="_blank">http://mysql.taobao.org/monthly/</a><br>全棧技術(shù):<a href="https://links.jianshu.com/go?to=https%3A%2F%2Fblog.thankbabe.com%2F" target="_blank">https://blog.thankbabe.com/</a><br>美團(tuán)技術(shù)博客:<a href="https://links.jianshu.com/go?to=https%3A%2F%2Ftech.meituan.com%2F" target="_blank">https://tech.meituan.com/</a><br>阿里中間件團(tuán)隊(duì)博客:<a href="https://links.jianshu.com/go?to=http%3A%2F%2Fjm.taobao.org%2F" target="_blank">http://jm.taobao.org/</a><br>運(yùn)維社區(qū):<a href="https://links.jianshu.com/go?to=https%3A%2F%2Fwww.unixhot.com%2F" target="_blank">https://www.unixhot.com/</a><br>cacheCloud:<a href="https://links.jianshu.com/go?to=https%3A%2F%2Fcachecloud.github.io%2F" target="_blank">https://cachecloud.github.io/</a><br>go知識(shí)圖譜:<a href="https://links.jianshu.com/go?to=https%3A%2F%2Fwww.processon.com%2Fview%2Flink%2F5a9ba4c8e4b0a9d22eb3bdf0" target="_blank">https://www.processon.com/view/link/5a9ba4c8e4b0a9d22eb3bdf0</a><br>go語言分析 <a href="https://links.jianshu.com/go?to=https%3A%2F%2Fgithub.com%2Fchangkun%2Fgo-under-the-hood%2Fblob%2Fmaster%2Fbook%2Fzh-cn%2FTOC.md" target="_blank">https://github.com/changkun/go-under-the-hood/blob/master/book/zh-cn/TOC.md</a><br>后端架構(gòu)體系:<a href="https://links.jianshu.com/go?to=https%3A%2F%2Fgithub.com%2Fxingshaocheng%2Farchitect-awesome" target="_blank">https://github.com/xingshaocheng/architect-awesome</a><br>架構(gòu)師技術(shù)圖譜:<a href="https://links.jianshu.com/go?to=https%3A%2F%2Fgithub.com%2Ftoutiaoio%2Fawesome-architecture" target="_blank">https://github.com/toutiaoio/awesome-architecture</a><br>互聯(lián)網(wǎng) Java 工程師進(jìn)階知識(shí)完全掃盲:<a href="https://links.jianshu.com/go?to=https%3A%2F%2Fgithub.com%2Fdoocs%2Fadvanced-java" target="_blank">https://github.com/doocs/advanced-java</a><br>從Docker到Kubernetes進(jìn)階:<a href="https://links.jianshu.com/go?to=https%3A%2F%2Fwww.qikqiak.com%2Fk8s-book%2F" target="_blank">https://www.qikqiak.com/k8s-book/</a>
