Clickhouse單機(jī)版快速入門

本文以操作系統(tǒng)為macos為例
參考鏈接 https://clickhouse.tech/docs/zh/getting-started/tutorial/

macos下載二進(jìn)制包

curl -O 'https://builds.clickhouse.tech/master/macos/clickhouse' && chmod a+x ./clickhouse

創(chuàng)建config.xml和users.xml
文件的內(nèi)容如下

https://github.com/ClickHouse/ClickHouse/blob/master/programs/server/config.xml

https://github.com/ClickHouse/ClickHouse/blob/master/programs/server/users.xml

啟動(dòng)服務(wù)端和客戶端

修改文件句柄數(shù)

如果文件句柄數(shù)設(shè)置的比較小锹漱，client或者server端在運(yùn)行中可能會報(bào)too many open files xxx的錯(cuò)。在運(yùn)行server和client的終端都運(yùn)行一下

ulimit -n 10240

ulimit -a # 查看文件句柄數(shù)是否修改成功

啟動(dòng)服務(wù)端

在一個(gè)終端運(yùn)行

sudo ./clickhouse server --config-file=config.xml

啟動(dòng)客戶端

在另一個(gè)終端運(yùn)行

./clickhouse client

命令行clickhouse-client的快速提示

# 交互模式
clickhouse-client
clickhouse-client --host=... --port=... --user=... --password=...

# 多行查詢
clickhouse-client -m
clickhouse-client --multiline

# 批處理
clickhouse-client --query='SELECT 1'
echo 'SELECT 1' | clickhouse-client
clickhouse-client <<< 'SELECT 1'

# 從指定文件中插入數(shù)據(jù)
clickhouse-client --query='INSERT INTO table VALUES' < data.txt
clickhouse-client --query='INSERT INTO table FORMAT TabSeparated' < data.tsv

使用DataGrip連接clickhouse

下載并且提取數(shù)據(jù)

下載數(shù)據(jù)集

下載完后解壓沿猜，解壓完后兩個(gè)文件大概是10G的大小

https://datasets.clickhouse.tech/visits/tsv/visits_v1.tsv.xz
https://datasets.clickhouse.tech/hits/tsv/hits_v1.tsv.xz

創(chuàng)建數(shù)據(jù)庫

CREATE DATABASE IF NOT EXISTS tutorial;

創(chuàng)建表引擎

與創(chuàng)建數(shù)據(jù)庫相比玫锋，創(chuàng)建表的語法要復(fù)雜得多（請參閱參考資料. 一般CREATE TABLE聲明必須指定三個(gè)關(guān)鍵的事情:

要?jiǎng)?chuàng)建的表的名稱弄跌。
表結(jié)構(gòu)亭畜，例如：列名和對應(yīng)的數(shù)據(jù)類型劲件。
表引擎及其設(shè)置唇敞，這決定了對此表的查詢操作是如何在物理層面執(zhí)行的所有細(xì)節(jié)。

創(chuàng)建hits_v1表

CREATE TABLE tutorial.hits_v1
(
    `WatchID` UInt64,
    `JavaEnable` UInt8,
    `Title` String,
    `GoodEvent` Int16,
    `EventTime` DateTime,
    `EventDate` Date,
    `CounterID` UInt32,
    `ClientIP` UInt32,
    `ClientIP6` FixedString(16),
    `RegionID` UInt32,
    `UserID` UInt64,
    `CounterClass` Int8,
    `OS` UInt8,
    `UserAgent` UInt8,
    `URL` String,
    `Referer` String,
    `URLDomain` String,
    `RefererDomain` String,
    `Refresh` UInt8,
    `IsRobot` UInt8,
    `RefererCategories` Array(UInt16),
    `URLCategories` Array(UInt16),
    `URLRegions` Array(UInt32),
    `RefererRegions` Array(UInt32),
    `ResolutionWidth` UInt16,
    `ResolutionHeight` UInt16,
    `ResolutionDepth` UInt8,
    `FlashMajor` UInt8,
    `FlashMinor` UInt8,
    `FlashMinor2` String,
    `NetMajor` UInt8,
    `NetMinor` UInt8,
    `UserAgentMajor` UInt16,
    `UserAgentMinor` FixedString(2),
    `CookieEnable` UInt8,
    `JavascriptEnable` UInt8,
    `IsMobile` UInt8,
    `MobilePhone` UInt8,
    `MobilePhoneModel` String,
    `Params` String,
    `IPNetworkID` UInt32,
    `TraficSourceID` Int8,
    `SearchEngineID` UInt16,
    `SearchPhrase` String,
    `AdvEngineID` UInt8,
    `IsArtifical` UInt8,
    `WindowClientWidth` UInt16,
    `WindowClientHeight` UInt16,
    `ClientTimeZone` Int16,
    `ClientEventTime` DateTime,
    `SilverlightVersion1` UInt8,
    `SilverlightVersion2` UInt8,
    `SilverlightVersion3` UInt32,
    `SilverlightVersion4` UInt16,
    `PageCharset` String,
    `CodeVersion` UInt32,
    `IsLink` UInt8,
    `IsDownload` UInt8,
    `IsNotBounce` UInt8,
    `FUniqID` UInt64,
    `HID` UInt32,
    `IsOldCounter` UInt8,
    `IsEvent` UInt8,
    `IsParameter` UInt8,
    `DontCountHits` UInt8,
    `WithHash` UInt8,
    `HitColor` FixedString(1),
    `UTCEventTime` DateTime,
    `Age` UInt8,
    `Sex` UInt8,
    `Income` UInt8,
    `Interests` UInt16,
    `Robotness` UInt8,
    `GeneralInterests` Array(UInt16),
    `RemoteIP` UInt32,
    `RemoteIP6` FixedString(16),
    `WindowName` Int32,
    `OpenerName` Int32,
    `HistoryLength` Int16,
    `BrowserLanguage` FixedString(2),
    `BrowserCountry` FixedString(2),
    `SocialNetwork` String,
    `SocialAction` String,
    `HTTPError` UInt16,
    `SendTiming` Int32,
    `DNSTiming` Int32,
    `ConnectTiming` Int32,
    `ResponseStartTiming` Int32,
    `ResponseEndTiming` Int32,
    `FetchTiming` Int32,
    `RedirectTiming` Int32,
    `DOMInteractiveTiming` Int32,
    `DOMContentLoadedTiming` Int32,
    `DOMCompleteTiming` Int32,
    `LoadEventStartTiming` Int32,
    `LoadEventEndTiming` Int32,
    `NSToDOMContentLoadedTiming` Int32,
    `FirstPaintTiming` Int32,
    `RedirectCount` Int8,
    `SocialSourceNetworkID` UInt8,
    `SocialSourcePage` String,
    `ParamPrice` Int64,
    `ParamOrderID` String,
    `ParamCurrency` FixedString(3),
    `ParamCurrencyID` UInt16,
    `GoalsReached` Array(UInt32),
    `OpenstatServiceName` String,
    `OpenstatCampaignID` String,
    `OpenstatAdID` String,
    `OpenstatSourceID` String,
    `UTMSource` String,
    `UTMMedium` String,
    `UTMCampaign` String,
    `UTMContent` String,
    `UTMTerm` String,
    `FromTag` String,
    `HasGCLID` UInt8,
    `RefererHash` UInt64,
    `URLHash` UInt64,
    `CLID` UInt32,
    `YCLID` UInt64,
    `ShareService` String,
    `ShareURL` String,
    `ShareTitle` String,
    `ParsedParams` Nested(
        Key1 String,
        Key2 String,
        Key3 String,
        Key4 String,
        Key5 String,
        ValueDouble Float64),
    `IslandID` FixedString(16),
    `RequestNum` UInt32,
    `RequestTry` UInt8
)
ENGINE = MergeTree()
PARTITION BY toYYYYMM(EventDate)
ORDER BY (CounterID, EventDate, intHash32(UserID))
SAMPLE BY intHash32(UserID)

創(chuàng)建visits_v1表

CREATE TABLE tutorial.visits_v1
(
    `CounterID` UInt32,
    `StartDate` Date,
    `Sign` Int8,
    `IsNew` UInt8,
    `VisitID` UInt64,
    `UserID` UInt64,
    `StartTime` DateTime,
    `Duration` UInt32,
    `UTCStartTime` DateTime,
    `PageViews` Int32,
    `Hits` Int32,
    `IsBounce` UInt8,
    `Referer` String,
    `StartURL` String,
    `RefererDomain` String,
    `StartURLDomain` String,
    `EndURL` String,
    `LinkURL` String,
    `IsDownload` UInt8,
    `TraficSourceID` Int8,
    `SearchEngineID` UInt16,
    `SearchPhrase` String,
    `AdvEngineID` UInt8,
    `PlaceID` Int32,
    `RefererCategories` Array(UInt16),
    `URLCategories` Array(UInt16),
    `URLRegions` Array(UInt32),
    `RefererRegions` Array(UInt32),
    `IsYandex` UInt8,
    `GoalReachesDepth` Int32,
    `GoalReachesURL` Int32,
    `GoalReachesAny` Int32,
    `SocialSourceNetworkID` UInt8,
    `SocialSourcePage` String,
    `MobilePhoneModel` String,
    `ClientEventTime` DateTime,
    `RegionID` UInt32,
    `ClientIP` UInt32,
    `ClientIP6` FixedString(16),
    `RemoteIP` UInt32,
    `RemoteIP6` FixedString(16),
    `IPNetworkID` UInt32,
    `SilverlightVersion3` UInt32,
    `CodeVersion` UInt32,
    `ResolutionWidth` UInt16,
    `ResolutionHeight` UInt16,
    `UserAgentMajor` UInt16,
    `UserAgentMinor` UInt16,
    `WindowClientWidth` UInt16,
    `WindowClientHeight` UInt16,
    `SilverlightVersion2` UInt8,
    `SilverlightVersion4` UInt16,
    `FlashVersion3` UInt16,
    `FlashVersion4` UInt16,
    `ClientTimeZone` Int16,
    `OS` UInt8,
    `UserAgent` UInt8,
    `ResolutionDepth` UInt8,
    `FlashMajor` UInt8,
    `FlashMinor` UInt8,
    `NetMajor` UInt8,
    `NetMinor` UInt8,
    `MobilePhone` UInt8,
    `SilverlightVersion1` UInt8,
    `Age` UInt8,
    `Sex` UInt8,
    `Income` UInt8,
    `JavaEnable` UInt8,
    `CookieEnable` UInt8,
    `JavascriptEnable` UInt8,
    `IsMobile` UInt8,
    `BrowserLanguage` UInt16,
    `BrowserCountry` UInt16,
    `Interests` UInt16,
    `Robotness` UInt8,
    `GeneralInterests` Array(UInt16),
    `Params` Array(String),
    `Goals` Nested(
        ID UInt32,
        Serial UInt32,
        EventTime DateTime,
        Price Int64,
        OrderID String,
        CurrencyID UInt32),
    `WatchIDs` Array(UInt64),
    `ParamSumPrice` Int64,
    `ParamCurrency` FixedString(3),
    `ParamCurrencyID` UInt16,
    `ClickLogID` UInt64,
    `ClickEventID` Int32,
    `ClickGoodEvent` Int32,
    `ClickEventTime` DateTime,
    `ClickPriorityID` Int32,
    `ClickPhraseID` Int32,
    `ClickPageID` Int32,
    `ClickPlaceID` Int32,
    `ClickTypeID` Int32,
    `ClickResourceID` Int32,
    `ClickCost` UInt32,
    `ClickClientIP` UInt32,
    `ClickDomainID` UInt32,
    `ClickURL` String,
    `ClickAttempt` UInt8,
    `ClickOrderID` UInt32,
    `ClickBannerID` UInt32,
    `ClickMarketCategoryID` UInt32,
    `ClickMarketPP` UInt32,
    `ClickMarketCategoryName` String,
    `ClickMarketPPName` String,
    `ClickAWAPSCampaignName` String,
    `ClickPageName` String,
    `ClickTargetType` UInt16,
    `ClickTargetPhraseID` UInt64,
    `ClickContextType` UInt8,
    `ClickSelectType` Int8,
    `ClickOptions` String,
    `ClickGroupBannerID` Int32,
    `OpenstatServiceName` String,
    `OpenstatCampaignID` String,
    `OpenstatAdID` String,
    `OpenstatSourceID` String,
    `UTMSource` String,
    `UTMMedium` String,
    `UTMCampaign` String,
    `UTMContent` String,
    `UTMTerm` String,
    `FromTag` String,
    `HasGCLID` UInt8,
    `FirstVisit` DateTime,
    `PredLastVisit` Date,
    `LastVisit` Date,
    `TotalVisits` UInt32,
    `TraficSource` Nested(
        ID Int8,
        SearchEngineID UInt16,
        AdvEngineID UInt8,
        PlaceID UInt16,
        SocialSourceNetworkID UInt8,
        Domain String,
        SearchPhrase String,
        SocialSourcePage String),
    `Attendance` FixedString(16),
    `CLID` UInt32,
    `YCLID` UInt64,
    `NormalizedRefererHash` UInt64,
    `SearchPhraseHash` UInt64,
    `RefererDomainHash` UInt64,
    `NormalizedStartURLHash` UInt64,
    `StartURLDomainHash` UInt64,
    `NormalizedEndURLHash` UInt64,
    `TopLevelDomain` UInt64,
    `URLScheme` UInt64,
    `OpenstatServiceNameHash` UInt64,
    `OpenstatCampaignIDHash` UInt64,
    `OpenstatAdIDHash` UInt64,
    `OpenstatSourceIDHash` UInt64,
    `UTMSourceHash` UInt64,
    `UTMMediumHash` UInt64,
    `UTMCampaignHash` UInt64,
    `UTMContentHash` UInt64,
    `UTMTermHash` UInt64,
    `FromHash` UInt64,
    `WebVisorEnabled` UInt8,
    `WebVisorActivity` UInt32,
    `ParsedParams` Nested(
        Key1 String,
        Key2 String,
        Key3 String,
        Key4 String,
        Key5 String,
        ValueDouble Float64),
    `Market` Nested(
        Type UInt8,
        GoalID UInt32,
        OrderID String,
        OrderPrice Int64,
        PP UInt32,
        DirectPlaceID UInt32,
        DirectOrderID UInt32,
        DirectBannerID UInt32,
        GoodID String,
        GoodName String,
        GoodQuantity Int32,
        GoodPrice Int64),
    `IslandID` FixedString(16)
)
ENGINE = CollapsingMergeTree(Sign)
PARTITION BY toYYYYMM(StartDate)
ORDER BY (CounterID, StartDate, intHash32(UserID), VisitID)
SAMPLE BY intHash32(UserID)

hits_v1使用 MergeTree引擎启昧，而visits_v1使用 Collapsing引擎

導(dǎo)入數(shù)據(jù)

從文件中導(dǎo)入數(shù)據(jù)

./clickhouse client --query "INSERT INTO tutorial.hits_v1 FORMAT TSV" --max_insert_block_size=100000 < hits_v1.tsv
./clickhouse client --query "INSERT INTO tutorial.visits_v1 FORMAT TSV" --max_insert_block_size=100000 < visits_v1.tsv

查看默認(rèn)配置

SELECT name, value, changed, description
FROM system.settings
WHERE name LIKE '%max_insert_b%'

OPTIMIZE導(dǎo)入后的表

optimize table tutorial.hits_v1 final
optimize table tutorial.visits_v1 final

檢查數(shù)據(jù)是否插入成功

./clickhouse client --query "SELECT COUNT(*) FROM tutorial.hits_v1"
./clickhouse client --query "SELECT COUNT(*) FROM tutorial.visits_v1"

查詢數(shù)據(jù)

可以在DataGrip中使用

截屏2021-04-21 下午3.42.45.png

select StartURL      as URL,
       avg(Duration) as AvgDuration
from tutorial.visits_v1
where StartDate between '2014-03-23' and '2014-03-30'
group by URL
ORDER BY AvgDuration DESC
limit 10

SELECT sum(Sign)                           AS visits,
       sumIf(Sign, has(Goals.ID, 1105530)) AS goal_visits,
       (100. * goal_visits) / visits       AS goal_percent
FROM tutorial.visits_v1
WHERE (CounterID = 912887)
  AND (toYYYYMM(StartDate) = 201403)
  AND (domain(StartURL) = 'yandex.ru')

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者

人面猴
序言：七十年代末叙凡，一起剝皮案震驚了整個(gè)濱河市，隨后出現(xiàn)的幾起案子密末，更是在濱河造成了極大的恐慌握爷，老刑警劉巖，帶你破解...
沈念sama閱讀 206,968評論 6贊 482
死咒
序言：濱河連續(xù)發(fā)生了三起死亡事件严里，死亡現(xiàn)場離奇詭異新啼，居然都是意外死亡，警方通過查閱死者的電腦和手機(jī)刹碾，發(fā)現(xiàn)死者居然都...
沈念sama閱讀 88,601評論 2贊 382
救了他兩次的神仙讓他今天三更去死
文/潘曉璐我一進(jìn)店門师抄，熙熙樓的掌柜王于貴愁眉苦臉地迎上來，“玉大人教硫，你說我怎么就攤上這事叨吮。” “怎么了瞬矩？”我有些...
開封第一講書人閱讀 153,220評論 0贊 344
道士緝兇錄：失蹤的賣姜人
文/不壞的土叔我叫張陵茶鉴，是天一觀的道長。經(jīng)常有香客問我景用，道長涵叮，這世上最難降的妖魔是什么？我笑而不...
開封第一講書人閱讀 55,416評論 1贊 279
?港島之戀（遺憾婚禮）
正文為了忘掉前任伞插，我火速辦了婚禮割粮，結(jié)果婚禮上，老公的妹妹穿的比我還像新娘媚污。我一直安慰自己舀瓢，他們只是感情好，可當(dāng)我...
茶點(diǎn)故事閱讀 64,425評論 5贊 374
惡毒庶女頂嫁案：這布局不是一般人想出來的
文/花漫我一把揭開白布耗美。她就那樣靜靜地躺著京髓，像睡著了一般。火紅的嫁衣襯著肌膚如雪商架。梳的紋絲不亂的頭發(fā)上堰怨，一...
開封第一講書人閱讀 49,144評論 1贊 285
城市分裂傳說
那天，我揣著相機(jī)與錄音蛇摸，去河邊找鬼备图。笑死，一個(gè)胖子當(dāng)著我的面吹牛，可吹牛的內(nèi)容都是我干的揽涮。我是一名探鬼主播抠藕，決...
沈念sama閱讀 38,432評論 3贊 401
雙鴛鴦連環(huán)套：你想象不到人心有多黑
文/蒼蘭香墨我猛地睜開眼，長吁一口氣：“原來是場噩夢啊……” “哼绞吁！你這毒婦竟也來了幢痘？” 一聲冷哼從身側(cè)響起，我...
開封第一講書人閱讀 37,088評論 0贊 261
萬榮殺人案實(shí)錄
序言：老撾萬榮一對情侶失蹤家破，失蹤者是張志新（化名）和其女友劉穎颜说，沒想到半個(gè)月后，有當(dāng)?shù)厝嗽跇淞掷锇l(fā)現(xiàn)了一具尸體汰聋，經(jīng)...
沈念sama閱讀 43,586評論 1贊 300
?護(hù)林員之死
正文獨(dú)居荒郊野嶺守林人離奇死亡门粪，尸身上長有42處帶血的膿包…… 初始之章·張勛以下內(nèi)容為張勛視角年9月15日...
茶點(diǎn)故事閱讀 36,028評論 2贊 325
?白月光啟示錄
正文我和宋清朗相戀三年，在試婚紗的時(shí)候發(fā)現(xiàn)自己被綠了烹困。大學(xué)時(shí)的朋友給我發(fā)了我未婚夫和他白月光在一起吃飯的照片玄妈。...
茶點(diǎn)故事閱讀 38,137評論 1贊 334
活死人
序言：一個(gè)原本活蹦亂跳的男人離奇死亡，死狀恐怖髓梅，靈堂內(nèi)的尸體忽然破棺而出拟蜻，到底是詐尸還是另有隱情，我是刑警寧澤枯饿，帶...
沈念sama閱讀 33,783評論 4贊 324
?日本核電站爆炸內(nèi)幕
正文年R本政府宣布酝锅，位于F島的核電站，受9級特大地震影響奢方，放射性物質(zhì)發(fā)生泄漏搔扁。R本人自食惡果不足惜，卻給世界環(huán)境...
茶點(diǎn)故事閱讀 39,343評論 3贊 307
男人毒藥：我在死后第九天來索命
文/蒙蒙一蟋字、第九天我趴在偏房一處隱蔽的房頂上張望稿蹲。院中可真熱鬧，春花似錦鹊奖、人聲如沸苛聘。這莊子的主人今日做“春日...
開封第一講書人閱讀 30,333評論 0贊 19
一樁弒父案嫉入，背后竟有這般陰謀
文/蒼蘭香墨我抬頭看了看天上的太陽焰盗。三九已至，卻和暖如春咒林，著一層夾襖步出監(jiān)牢的瞬間，已是汗流浹背爷光。一陣腳步聲響...
開封第一講書人閱讀 31,559評論 1贊 262
情欲美人皮
我被黑心中介騙來泰國打工垫竞，沒想到剛下飛機(jī)就差點(diǎn)兒被人妖公主榨干…… 1. 我叫王不留，地道東北人。一個(gè)月前我還...
沈念sama閱讀 45,595評論 2贊 355
代替公主和親
正文我出身青樓欢瞪，卻偏偏與公主長得像活烙，于是被迫代替她去往敵國和親。傳聞我的和親對象是個(gè)殘疾皇子遣鼓，可洞房花燭夜當(dāng)晚...
茶點(diǎn)故事閱讀 42,901評論 2贊 345