【譯】保護 Consul 在特定設置中免受 RCE 風險的影響

2018年11月27日 Consul 團隊

介紹

咱們最近注意到了一組惡意軟件，它們主要針對具備容許遠程執行代碼這一特定配置的 Consul nodes 。 咱們的社區成員也 (負責任地) 報告了此惡意軟件形成的事件, 並與咱們合做, 在最近版本的 Consul 中添加了一個補丁, 以保護咱們免受這種威脅。

這篇文章詳細介紹了此惡意軟件可能會如何影響用戶, 具體取決於他們的配置, 並概述了咱們已經採起的步驟, 來提供支持1.2.四、1.1.一、1.0.8 和0.9.4 版本的補丁, 使舊版本的 Consul 在沒有重大版本升級的狀況下也能很容易受到保護。

總結

若是您已將-enable-script-checks設置爲 true", 或者正在運行 Consul 0.9.0 或更早版本, 而且Consul api 可經過網絡訪問的接口使用, 則應執行如下操做。

補救步驟:

1. 升級到下面連接的版本之一。
2. 若是須要腳本檢查, 請將 Consul 代理上的 "-enable-script-checks" 更改成 "-enable-local-script-checks。
3. 禁用 Consul 服務器上的腳本檢查。
4. 確保 Consul HTTP API 綁定到 loopback 接口, 而不是一個可公開訪問的接口。
5. 若是還沒有啓用, 則啓用 ACL 。

有關詳細信息, 請繼續閱讀下面的內容。

如何利用腳本檢查

腳本檢查是 Consul 能夠執行的健康檢查的一種類型，用於肯定目標服務的健康狀態。支持的檢查類型包括 HTTP、TCP、gRPC、Docker、Alias、TTL 和腳本檢查。檢查能夠經過 Consul API 註冊, 也能夠由 Consul 代理從檢查定義目錄中加載。

腳本檢查將在配置的時間間隔內執行 Consul 進程的任何命令或內聯腳本。Docker 檢查是相同的, 但經過 Docker daemon 的 API 在正在運行的 Docker 容器中執行腳本。默認狀況下, 做爲 Consul 0.9.0 的 Consul 安裝中禁用腳本 (和 Docker) 檢查, 儘管它們在許多狀況下很是有用, 但存在潛在的安全風險。

Consul API 是用於與 Consul 代理交互的 HTTP 接口, 包括註冊上面所述的運行情況檢查定義。在大多數狀況下, 咱們建議將此 API 綁定到 loopback 接口, 以防止意外暴露。咱們提供了一個普遍的 ACLl 系統, 可用於保護 API 交互, 包括註冊檢查。

使代理容易受到這種攻擊的必要條件是:

• API 可經過網絡訪問的接口使用。
• 腳本檢查已啓用。
• ACL 被禁用或 ACL Token 被破壞。

在上述條件下, 攻擊者能夠對遠程代理註冊具備惡意負載檢查。根據設計, 腳本檢查容許任意執行代碼, 所以容許經過公開的 API 啓用檢查的服務註冊會帶來 RCE (遠程代碼執行) 威脅。

Consul的早期設計假定HTTP API僅限於受信任的本地網絡訪問，直到後來才清楚許多用戶因爲各類緣由不知道（或仍然選擇）公開它, 這促使在 0.9.0 版本中更改成在默認狀況下禁用腳本檢查。

建議將啓用 ACL 做爲最佳作法, 但即便在限制性 ACL 到位的狀況下, 若是攻擊者用至少一個能夠註冊服務的 ACL 損害主機，則相同的ACL Token 將容許攻擊者在任何公開的遠程節點上註冊腳本檢查，由於根據設計，ACL的範圍不限於單個代理。咱們還沒有發現惡意軟件有利用本地可用的 ACL Token 針對受 ACL 保護的羣集, 但主動指出此風險但願使用者們注意。

已知的影響

上個月咱們有幾份關於這個問題的報告, 並證明它正在變得更加廣泛。咱們要確保咱們的 Consul 用戶社區意識到這一威脅, 並知道如何保護本身免受這種攻擊。

在每種已知狀況下, 攻擊者都經過 Consul 以外的一些不相關漏洞得到了對數據中心中一臺主機的訪問權限。而後, 攻擊者在網絡上檢測到暴露的 Consul API (可能經過自動掃描), 並使用腳本檢查將惡意軟件傳播到其餘服務器。

修復

許多用戶對其基礎結構中的服務使用腳本檢查功能, 所以徹底禁用腳本檢查將禁用他們所依賴的功能。

在本月初發布的領Consul 1.3.0 中, Consul 社區的一名成員提供了一個補丁, 添加了一個新的配置選項-- -enable-local-script-checks, 它容許註冊腳本檢查僅經過本地配置文件, 從而防止使用 HTTP API 註冊惡意檢查。咱們建議全部須要腳本檢查的用戶切換到此選項, 並找到避免的方法, 具體取決於對代理的遠程 API 訪問。

除此以外，咱們建議檢查 ACL 系統和 Consul API 正在偵聽的接口的使用。默認狀況下腳本檢查仍然徹底禁用。

考慮到經過多個主要版本更改升級到 1.3.0 (在那裏發佈了此功能) 的挑戰, 咱們支持-enable-local-script-checks " 選項, 併發布了多個修補版本的 Consul, 以確保咱們的用戶能夠以最小的干擾和努力來緩解這一威脅:

• 1.2.4
  • 更改日誌連接
  • 下載連接
• 1.1.1
  • 更改日誌連接
  • 下載連接
• 1.0.8
  • 更改日誌連接
  • 下載連接
• 0.9.4
  • 更改日誌連接
  • 下載連接

這些是該系列中的用戶操做版本的替代。若是您當前使用的是 0.9.0 以前的某個版本, 咱們建議升級到 0.9.4。與往常同樣, 咱們建議您在全部升級中遵循咱們的升級指南。版本1.3.0 及之後已包含此功能。同一版本的企業版也已可用。

一般, 咱們更喜歡看到用戶升級, 不多出現這樣的 backport 修復。因爲此威脅的性質, 咱們決定支持此修復程序, 以便用戶能夠快速保護本身, 而不須要可能帶來重大更改的主要版本升級。

依賴經過 HTTP API 註冊腳本檢查的用戶能夠繼續使用-enable-script-checks, 但應格外當心地啓用 ACL, 並確保在網絡上沒法訪問 API 偵聽器。咱們建議儘快找到不須要遠程 HTTP API 訪問的替代部署模式。

在 Consul 服務器上容許腳本檢查尤爲危險, 特別是爲了全局工具或UI訪問，在服務器上公開API是很常見的。服務器上的遠程可註冊腳本檢查將面臨從網絡上的任何節點向攻擊者暴露包括 ACL Token 在內的全部 Consul 狀態的風險。咱們正在考慮在將來版本的 Consul 中將在配置服務器上的腳本檢查變成一個很難犯的錯誤。一般，Consul服務器不管如何都不會運行其餘工做負載，所以禁用它們上的腳本檢查不太可能會致使問題，可是請檢查您的配置已確認沒有啓用它們，由於可能有其餘客戶端須要。

咱們還在探索其餘一些選項，以幫助通知用戶這種潛在的不安全配置，並考慮對將來版本的腳本檢查提供額外的保護。咱們爲腳本檢查和代理配置在文檔中添加了更加突出的警告，, 並在安全模型中添加了一個新的部分, 突出顯示了配置 Consul 的已知不安全的方式。爲了使運行中的 Consul 更易於安全訪問, 咱們在 Consul 1.4.0 中從新設計了ACL系統的接口,而且建議當前沒有使用 ACL 的 Consul 用戶使用這些接口。

--------------------------------------------------------------------

【原文】Protecting Consul from RCE Risk in Specific Configurations

NOV 27 2018 THE CONSUL TEAM

Introduction

We’ve recently become aware of a set of malware targeting Consul nodes with a specific configuration which allows remote code execution. Members of our community also (responsibly) reported incidents caused by this malware, and worked with us to include a patch in a recent version of Consul that protects from this threat in the wild.

This post details how this malware may affect users, depending on their configuration, as well as outlines the steps we've taken to backport a patch for versions 1.2.4, 1.1.1, 1.0.8, and 0.9.4 to make it easy for older versions of Consul to be secured without a major version upgrade.

Summary

You should take action if you have -enable-script-checks set to true, or are running Consul 0.9.0 or earlier, and the Consul API is available on an interface that can be accessed over the network.

Steps to remediate:

1. Upgrade to one of the versions linked below.
2. Change -enable-script-checks to -enable-local-script-checks on Consul agents if script checks are required.
3. Disable script checks on Consul servers.
4. Ensure the Consul HTTP API is bound to a loopback interface instead of one publicly accessible.
5. Enable ACLs if not already enabled.

For full details, continue reading below.

How Script Checks Can Be Exploited

Script checks are one type of health check Consul can execute to determine the health status of a target service. Supported check types are HTTP, TCP, gRPC, Docker, Alias, TTL, and script checks. Checks can be registered via the Consul API or loaded from a directory of check definitions by the Consul agent.

Script checks will execute any command or inline script by the Consul process at the configured interval. Docker checks are the same, but execute the script within a running Docker container via the Docker daemon's API. Script (and Docker) checks are disabled by default in Consul installations as of Consul 0.9.0, and though they are useful for many cases present a potential security risk.

The Consul API is the HTTP interface for interacting with the Consul agent, including registering health check definitions as noted above. We recommend binding this API to the loopback interface in the majority of cases to prevent accidental exposure. We provide an extensive ACL system that can be utilized to protect API interactions, including registering checks.

The necessary conditions that make an agent vulnerable to this attack are:

• The API is available on an interface that can be accessed over the network.
• Script checks are enabled.
• ACLs are disabled or an ACL token is compromised.

Given the above conditions, an attacker can register a check on a remote agent with a malicious payload. By design, script checks allow arbitrary code execution, so allowing service registration with checks enabled via an exposed API presents an RCE (remote code execution) threat.

Consul's early design assumed that HTTP API was restricted to trusted local network access, and only later did it become clear that many users were not aware (or still chose) to expose it for a variety of reasons, prompting the change to disable script checks by default in 0.9.0.

Enabling ACLs is recommended as a best practice, but even with restrictive ACLs in place, if an attacker compromises a host with at least one ACL present that can register a service, that same ACL token will allow the attacker to register with a script check on any remote node that is exposed, since ACLs by design are not scoped to a single agent. We are not aware of malware exploiting locally-available ACL tokens against an ACL-protected cluster yet, but wish to proactively point out this risk.

Known Impact

We've had several reports of this in the last month and have confirmed that it is becoming more widespread. We want to ensure our community of Consul users is aware of this threat and knows how to protect themselves against this type of attack.

In each known case, the attacker gained access to one host in the datacenter via some unrelated vulnerability outside of Consul. The attacker then detected exposed Consul APIs on the network (likely though automated scanning) and used script checks to propagate the malware to other servers.

Remediation

Many users use the script check feature for services across their infrastructure, so disabling script checks altogether would disable functionality they rely on.

In Consul 1.3.0, released earlier this month, a member of the Consul community contributed a patch that adds a new configuration option, -enable-local-script-checks, which allows script checks to be registered only via local configuration files, thus preventing use of the HTTP API to register malicious checks. We recommend all users who need script checks switch to this option and find ways to avoid depending on remote API access to agents.

In addition to this, we recommend reviewing usage of the ACL system and interfaces that the Consul API is listening on. Script checks remain disabled entirely by default.

Given the challenge of upgrading through multiple major version changes to 1.3.0, where this feature was released, we have backported the -enable-local-script-checks option and released multiple patched versions of Consul to ensure our users can mitigate this threat with minimum disruption and effort:

• 1.2.4
  • Changelog link
  • Download link
• 1.1.1
  • Changelog link
  • Download link
• 1.0.8
  • Changelog link
  • Download link
• 0.9.4
  • Changelog link
  • Download link

These are drop-in replacements for users operating versions in that series. If you are currently using a version before 0.9.0, we recommend upgrading to 0.9.4 - there may be some breaking changes from earlier versions, but these are minimal compared with the much larger changes made in 1.0.0. As always, we recommend following our upgrade guidefor all upgrades. Version 1.3.0 and onwards already contain this feature. Enterprise binaries for the same versions are also available.

In general, we prefer to see users upgrade, and rarely backport fixes like this. Because of the nature of this threat, we decided to backport this fix so that users could protect themselves quickly, without needing a major version upgrade that might potentially introduce breaking changes.

Users that rely on registering script checks via the HTTP API can continue to use -enable-script-checks, but should take extra care to enable ACLs and to ensure the API listener is not accessible on the network. We recommend finding alternative deployment patterns where remote HTTP API access is not required as soon as possible.

It is especially dangerous to allow script checks on Consul servers, especially since it's common to expose the API on servers for global tooling or UI access. Remotely registerable script checks on servers risk exposing all Consul state including ACL tokens to the attacker from any node on the network. We are considering making it a hard error to configure script checks on servers in future versions of Consul. Generally Consul servers tend not to run other workloads anyway so disabling script checks on them isn't likely to cause a problem, but please check your configuration didn't enable them just because other clients need to.

We are also exploring several other options to help notify users of this potentially insecure configuration, as well as considering additional protections for script checks for future releases. We added more prominent warnings to our documentation for script checks and agent configuration as well as adding a new section to our security modelhighlighting known-insecure ways to configure Consul. As part of making running Consul securely more accessible, we redesigned the interfaces to our ACL system in Consul 1.4.0 and recommend usage of that for users not currently using ACLs with Consul.