2018年11月27日 Consul 團隊
介紹
我們最近注意到了一組惡意軟件止吁，它們主要針對具有允許遠程執(zhí)行代碼這一特定配置的 Consul nodes 。 我們的社區(qū)成員也 (負責(zé)任地) 報告了此惡意軟件造成的事件, 并與我們合作, 在最近版本的 Consul 中添加了一個補丁, 以保護我們免受這種威脅制恍。
這篇文章詳細介紹了此惡意軟件可能會如何影響用戶, 具體取決于他們的配置, 并概述了我們已經(jīng)采取的步驟, 來提供支持1.2.4、1.1.1神凑、1.0.8 和0.9.4 版本的補丁, 使舊版本的 Consul 在沒有重大版本升級的情況下也能很容易受到保護净神。
總結(jié)
如果您已將-enable-script-checks設(shè)置為 true", 或者正在運行 Consul 0.9.0 或更早版本, 并且Consul api 可通過網(wǎng)絡(luò)訪問的接口使用, 則應(yīng)執(zhí)行以下操作。
補救步驟:
- 升級到下面鏈接的版本之一溉委。
- 如果需要腳本檢查, 請將 Consul 代理上的 "
-enable-script-checks" 更改為 "-enable-local-script-checks鹃唯。 - 禁用 Consul 服務(wù)器上的腳本檢查。
- 確保 Consul HTTP API 綁定到 loopback 接口, 而不是一個可公開訪問的接口薛躬。
- 如果尚未啟用, 則啟用 ACL 俯渤。
有關(guān)詳細信息, 請繼續(xù)閱讀下面的內(nèi)容。
如何利用腳本檢查
腳本檢查是 Consul 可以執(zhí)行的健康檢查的一種類型型宝，用于確定目標服務(wù)的健康狀態(tài)八匠。支持的檢查類型包括 HTTP、TCP趴酣、gRPC梨树、Docker、Alias岖寞、TTL 和腳本檢查抡四。檢查可以通過 Consul API 注冊, 也可以由 Consul 代理從檢查定義目錄中加載。
腳本檢查將在配置的時間間隔內(nèi)執(zhí)行 Consul 進程的任何命令或內(nèi)聯(lián)腳本。Docker 檢查是相同的, 但通過 Docker daemon 的 API 在正在運行的 Docker 容器中執(zhí)行腳本指巡。默認情況下, 作為 Consul 0.9.0 的 Consul 安裝中禁用腳本 (和 Docker) 檢查, 盡管它們在許多情況下非常有用, 但存在潛在的安全風(fēng)險淑履。
Consul API 是用于與 Consul 代理交互的 HTTP 接口, 包括注冊上面所述的運行狀況檢查定義。在大多數(shù)情況下, 我們建議將此 API 綁定到 loopback 接口, 以防止意外暴露藻雪。我們提供了一個廣泛的 ACLl 系統(tǒng), 可用于保護 API 交互, 包括注冊檢查秘噪。
使代理容易受到這種攻擊的必要條件是:
- API 可通過網(wǎng)絡(luò)訪問的接口使用。
- 腳本檢查已啟用勉耀。
- ACL 被禁用或 ACL Token 被破壞指煎。
在上述條件下, 攻擊者可以對遠程代理注冊具有惡意負載檢查。根據(jù)設(shè)計, 腳本檢查允許任意執(zhí)行代碼, 因此允許通過公開的 API 啟用檢查的服務(wù)注冊會帶來 RCE (遠程代碼執(zhí)行) 威脅便斥。
Consul的早期設(shè)計假定HTTP API僅限于受信任的本地網(wǎng)絡(luò)訪問至壤，直到后來才清楚許多用戶由于各種原因不知道（或仍然選擇）公開它, 這促使在 0.9.0 版本中更改為在默認情況下禁用腳本檢查。
建議將啟用 ACL 作為最佳做法, 但即使在限制性 ACL 到位的情況下, 如果攻擊者用至少一個可以注冊服務(wù)的 ACL 損害主機枢纠，則相同的ACL Token 將允許攻擊者在任何公開的遠程節(jié)點上注冊腳本檢查像街，因為根據(jù)設(shè)計，ACL的范圍不限于單個代理京郑。我們尚未發(fā)現(xiàn)惡意軟件有利用本地可用的 ACL Token 針對受 ACL 保護的群集, 但主動指出此風(fēng)險希望使用者們注意宅广。
已知的影響
上個月我們有幾份關(guān)于這個問題的報告, 并證實它正在變得更加普遍葫掉。我們要確保我們的 Consul 用戶社區(qū)意識到這一威脅, 并知道如何保護自己免受這種攻擊些举。
在每種已知情況下, 攻擊者都通過 Consul 之外的一些不相關(guān)漏洞獲得了對數(shù)據(jù)中心中一臺主機的訪問權(quán)限。然后, 攻擊者在網(wǎng)絡(luò)上檢測到暴露的 Consul API (可能通過自動掃描), 并使用腳本檢查將惡意軟件傳播到其他服務(wù)器俭厚。
修復(fù)
許多用戶對其基礎(chǔ)結(jié)構(gòu)中的服務(wù)使用腳本檢查功能, 因此完全禁用腳本檢查將禁用他們所依賴的功能户魏。
在本月初發(fā)布的領(lǐng)Consul 1.3.0 中, Consul 社區(qū)的一名成員提供了一個補丁, 添加了一個新的配置選項-- -enable-local-script-checks, 它允許注冊腳本檢查僅通過本地配置文件, 從而防止使用 HTTP API 注冊惡意檢查。我們建議所有需要腳本檢查的用戶切換到此選項, 并找到避免的方法, 具體取決于對代理的遠程 API 訪問挪挤。
除此之外叼丑，我們建議檢查 ACL 系統(tǒng)和 Consul API 正在偵聽的接口的使用。默認情況下腳本檢查仍然完全禁用扛门。
考慮到通過多個主要版本更改升級到 1.3.0 (在那里發(fā)布了此功能) 的挑戰(zhàn), 我們支持-enable-local-script-checks " 選項, 并發(fā)布了多個修補版本的 Consul, 以確保我們的用戶可以以最小的干擾和努力來緩解這一威脅:
這些是該系列中的用戶操作版本的替代鸠信。如果您當(dāng)前使用的是 0.9.0 之前的某個版本, 我們建議升級到 0.9.4。與往常一樣, 我們建議您在所有升級中遵循我們的升級指南论寨。版本1.3.0 及以后已包含此功能星立。同一版本的企業(yè)版也已可用。
通常, 我們更喜歡看到用戶升級, 很少出現(xiàn)這樣的 backport 修復(fù)葬凳。由于此威脅的性質(zhì), 我們決定支持此修復(fù)程序, 以便用戶可以快速保護自己, 而不需要可能帶來重大更改的主要版本升級绰垂。
依賴通過 HTTP API 注冊腳本檢查的用戶可以繼續(xù)使用-enable-script-checks, 但應(yīng)格外小心地啟用 ACL, 并確保在網(wǎng)絡(luò)上無法訪問 API 偵聽器。我們建議盡快找到不需要遠程 HTTP API 訪問的替代部署模式火焰。
在 Consul 服務(wù)器上允許腳本檢查尤其危險, 特別是為了全局工具或UI訪問劲装，在服務(wù)器上公開API是很常見的。服務(wù)器上的遠程可注冊腳本檢查將面臨從網(wǎng)絡(luò)上的任何節(jié)點向攻擊者暴露包括 ACL Token 在內(nèi)的所有 Consul 狀態(tài)的風(fēng)險。我們正在考慮在未來版本的 Consul 中將在配置服務(wù)器上的腳本檢查變成一個很難犯的錯誤占业。通常绒怨，Consul服務(wù)器無論如何都不會運行其他工作負載，因此禁用它們上的腳本檢查不太可能會導(dǎo)致問題谦疾，但是請檢查您的配置已確認沒有啟用它們窖逗，因為可能有其他客戶端需要。
我們還在探索其他一些選項餐蔬，以幫助通知用戶這種潛在的不安全配置碎紊，并考慮對未來版本的腳本檢查提供額外的保護。我們?yōu)槟_本檢查和代理配置在文檔中添加了更加突出的警告樊诺，, 并在安全模型中添加了一個新的部分, 突出顯示了配置 Consul 的已知不安全的方式仗考。為了使運行中的 Consul 更易于安全訪問, 我們在 Consul 1.4.0 中重新設(shè)計了ACL系統(tǒng)的接口,并且建議當(dāng)前沒有使用 ACL 的 Consul 用戶使用這些接口。
【原文】Protecting Consul from RCE Risk in Specific Configurations
NOV 27 2018 THE CONSUL TEAM
Introduction
We’ve recently become aware of a set of malware targeting Consul nodes with a specific configuration which allows remote code execution. Members of our community also (responsibly) reported incidents caused by this malware, and worked with us to include a patch in a recent version of Consul that protects from this threat in the wild.
This post details how this malware may affect users, depending on their configuration, as well as outlines the steps we've taken to backport a patch for versions 1.2.4, 1.1.1, 1.0.8, and 0.9.4 to make it easy for older versions of Consul to be secured without a major version upgrade.
Summary
You should take action if you have -enable-script-checks set to true, or are running Consul 0.9.0 or earlier, and the Consul API is available on an interface that can be accessed over the network.
Steps to remediate:
- Upgrade to one of the versions linked below.
- Change
-enable-script-checksto-enable-local-script-checkson Consul agents if script checks are required. - Disable script checks on Consul servers.
- Ensure the Consul HTTP API is bound to a loopback interface instead of one publicly accessible.
- Enable ACLs if not already enabled.
For full details, continue reading below.
How Script Checks Can Be Exploited
Script checks are one type of health check Consul can execute to determine the health status of a target service. Supported check types are HTTP, TCP, gRPC, Docker, Alias, TTL, and script checks. Checks can be registered via the Consul API or loaded from a directory of check definitions by the Consul agent.
Script checks will execute any command or inline script by the Consul process at the configured interval. Docker checks are the same, but execute the script within a running Docker container via the Docker daemon's API. Script (and Docker) checks are disabled by default in Consul installations as of Consul 0.9.0, and though they are useful for many cases present a potential security risk.
The Consul API is the HTTP interface for interacting with the Consul agent, including registering health check definitions as noted above. We recommend binding this API to the loopback interface in the majority of cases to prevent accidental exposure. We provide an extensive ACL system that can be utilized to protect API interactions, including registering checks.
The necessary conditions that make an agent vulnerable to this attack are:
- The API is available on an interface that can be accessed over the network.
- Script checks are enabled.
- ACLs are disabled or an ACL token is compromised.
Given the above conditions, an attacker can register a check on a remote agent with a malicious payload. By design, script checks allow arbitrary code execution, so allowing service registration with checks enabled via an exposed API presents an RCE (remote code execution) threat.
Consul's early design assumed that HTTP API was restricted to trusted local network access, and only later did it become clear that many users were not aware (or still chose) to expose it for a variety of reasons, prompting the change to disable script checks by default in 0.9.0.
Enabling ACLs is recommended as a best practice, but even with restrictive ACLs in place, if an attacker compromises a host with at least one ACL present that can register a service, that same ACL token will allow the attacker to register with a script check on any remote node that is exposed, since ACLs by design are not scoped to a single agent. We are not aware of malware exploiting locally-available ACL tokens against an ACL-protected cluster yet, but wish to proactively point out this risk.
Known Impact
We've had several reports of this in the last month and have confirmed that it is becoming more widespread. We want to ensure our community of Consul users is aware of this threat and knows how to protect themselves against this type of attack.
In each known case, the attacker gained access to one host in the datacenter via some unrelated vulnerability outside of Consul. The attacker then detected exposed Consul APIs on the network (likely though automated scanning) and used script checks to propagate the malware to other servers.
Remediation
Many users use the script check feature for services across their infrastructure, so disabling script checks altogether would disable functionality they rely on.
In Consul 1.3.0, released earlier this month, a member of the Consul community contributed a patch that adds a new configuration option, -enable-local-script-checks, which allows script checks to be registered only via local configuration files, thus preventing use of the HTTP API to register malicious checks. We recommend all users who need script checks switch to this option and find ways to avoid depending on remote API access to agents.
In addition to this, we recommend reviewing usage of the ACL system and interfaces that the Consul API is listening on. Script checks remain disabled entirely by default.
Given the challenge of upgrading through multiple major version changes to 1.3.0, where this feature was released, we have backported the -enable-local-script-checks option and released multiple patched versions of Consul to ensure our users can mitigate this threat with minimum disruption and effort:
- 1.2.4
- 1.1.1
- 1.0.8
- 0.9.4
These are drop-in replacements for users operating versions in that series. If you are currently using a version before 0.9.0, we recommend upgrading to 0.9.4 - there may be some breaking changes from earlier versions, but these are minimal compared with the much larger changes made in 1.0.0. As always, we recommend following our upgrade guidefor all upgrades. Version 1.3.0 and onwards already contain this feature. Enterprise binaries for the same versions are also available.
In general, we prefer to see users upgrade, and rarely backport fixes like this. Because of the nature of this threat, we decided to backport this fix so that users could protect themselves quickly, without needing a major version upgrade that might potentially introduce breaking changes.
Users that rely on registering script checks via the HTTP API can continue to use -enable-script-checks, but should take extra care to enable ACLs and to ensure the API listener is not accessible on the network. We recommend finding alternative deployment patterns where remote HTTP API access is not required as soon as possible.
It is especially dangerous to allow script checks on Consul servers, especially since it's common to expose the API on servers for global tooling or UI access. Remotely registerable script checks on servers risk exposing all Consul state including ACL tokens to the attacker from any node on the network. We are considering making it a hard error to configure script checks on servers in future versions of Consul. Generally Consul servers tend not to run other workloads anyway so disabling script checks on them isn't likely to cause a problem, but please check your configuration didn't enable them just because other clients need to.
We are also exploring several other options to help notify users of this potentially insecure configuration, as well as considering additional protections for script checks for future releases. We added more prominent warnings to our documentation for script checks and agent configuration as well as adding a new section to our security modelhighlighting known-insecure ways to configure Consul. As part of making running Consul securely more accessible, we redesigned the interfaces to our ACL system in Consul 1.4.0 and recommend usage of that for users not currently using ACLs with Consul.
