Vuls 漏洞掃描工具部署及效果展示

介紹

Vuls根據(jù)NVD提澎，OVAL等數(shù)據(jù)對主流Linux系統(tǒng)進行掃描，并具有完善的報告卸夕。

• 支持系統(tǒng)

  Distribution	Release
  Alpine	3.3 and later
  Ubuntu	14, 16, 18, 20
  Debian	8, 9, 10
  RHEL	5, 6, 7, 8
  Oracle Linux	5, 6, 7
  CentOS	6, 7, 8
  Amazon Linux	All
  FreeBSD	10, 11
  SUSE Enterprise	11, 12
  Raspbian	Jessie, Stretch, Buster

• 支持的數(shù)據(jù)庫

  Vulnerability Database:

  • NVD
  • JVN(Japanese)

  OVAL

  • Red Hat
  • Debian
  • Ubuntu
  • SUSE
  • Oracle Linux

  Security Advisory

  • Alpine-secdb
  • Red Hat Security Advisories
  • Debian Security Bug Tracker

  Commands(yum, zypper, pkg-audit)

  • RHSA / ALAS / ELSA / FreeBSD-SA
  • Changelog

  PoC, Exploit

  • Exploit Database
  • Metasploit-Framework modules

  CERT

  • US-CERT
  • JPCERT

  Libraries

  • Node.js Security Working Group
  • Ruby Advisory Database
  • Safety DB(Python)
  • PHP Security Advisories Database
  • RustSec Advisory Database

  WordPress

  • WPScan

更多詳情: 官網(wǎng)介紹

參考鏈接

1. Vuls官網(wǎng)手動安裝教程
2. Vuls安裝記錄
3. Vuls漏洞庫工具
4. 計算機漏洞和安全性評估語言 本地部署
5. Go Install Go語言安裝

安裝步驟

• 測試機器：CentOS

• 環(huán)境要求 Vuls requires the following packages.

  1. SQLite3, MySQL, PostgreSQL, Redis
  2. git
  3. gccGNU Make
  4. Go 必須安裝最新版Go逝慧，不然會有各種報錯

• 一. 環(huán)境部署：

  • 1.工具安裝

    $ sudo yum -y install sqlite git gcc make wget
    

  • 2.Go 安裝

    $ wget https://dl.google.com/go/go$latest_version.linux-amd64.tar.gz
    $ sudo tar -C /usr/local -xzf go$latest_version.linux-amd64.tar.gz
    $ mkdir $HOME/go
    

  • 3.配置Go環(huán)境變量

    $ vi  /etc/profile.d/goenv.sh
    

  • 4.在這個文件中添加以下環(huán)境變量

    export GOROOT=/usr/local/go 
    export GOPATH=$HOME/go 
    export PATH=$PATH:$GOROOT/bin:$GOPATH/bin
    

  • 5.設(shè)置環(huán)境變量到系統(tǒng)中

    $ source /etc/profile.d/goenv.sh
    

  • 6.檢測 go是否安裝成功

    $ go version
    

• 二. CVE庫部署

  • 1.從github下載部署工具并安裝

    $ sudo mkdir /var/log/vuls
    $ sudo chown <username> /var/log/vuls
    $ sudo chmod 700 /var/log/vuls
    $ mkdir -p $GOPATH/src/github.com/kotakanbe
    $ cd $GOPATH/src/github.com/kotakanbe
    $ git clone https://github.com/kotakanbe/go-cve-dictionary.git
    $ cd go-cve-dictionary
    $ make install
    

  • 2.獲取cve漏洞數(shù)據(jù) 大概需要一小時 1.6G 下載成功后可以看到sqlite數(shù)據(jù)庫 如果想縮短時間可以只獲取最近幾年的數(shù)據(jù)

    $ for i in `seq 2002 $(date +"%Y")`; do go-cve-dictionary fetchnvd -years $i; done
    

    [root@izbp1268h8zj95audt9fdez go-cve-dictionary]# ls -alh cve.sqlite3
    -rw-r--r-- 1 root root 1.6G Mar  1 19:04 cve.sqlite3
    

  • 踩坑

    • make install 失敗1:可以使用--debug查看更多信息

    • Permission denied 錯誤是文件權(quán)限不夠，沒有可執(zhí)行權(quán)限捌浩，提權(quán)即可

      [root@izbp1268h8zj95audt9fdez go-cve-dictionary]# make install
      GO111MODULE=on go install -ldflags "-X 'main.version=v0.5.8' -X 'main.revision=1aa7ea7'"
      /bin/sh: /usr/local/go/bin/go: Permission denied
      make: *** [install] Error 126
      [root@izbp1268h8zj95audt9fdez go-cve-dictionary]# make install --debug
      GNU Make 3.82
      Built for x86_64-redhat-linux-gnu
      Copyright (C) 2010  Free Software Foundation, Inc.
      License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
      Reading makefiles...
      Updating goal targets....
       File `install' does not exist.
      Must remake target `install'.
      Invoking recipe from GNUmakefile:31 to update target `install'.
      GO111MODULE=on go install -ldflags "-X 'main.version=v0.5.8' -X 'main.revision=1aa7ea7'"
      /bin/sh: /usr/local/go/bin/go: Permission denied
      

    • make install 失敗2: go 包安裝失敗 這是因為下載網(wǎng)址被墻了放刨，需要設(shè)置代理

      [root@izbp1268h8zj95audt9fdez go-cve-dictionary]# make install
      GO111MODULE=on go install -ldflags "-X 'main.version=v0.5.8' -X 'main.revision=1aa7ea7'"
      go: github.com/PuerkitoBio/goquery@v1.6.1: Get "https://proxy.golang.org/github.com/%21puerkito%21bio/goquery/@v/v1.6.1.mod": dial tcp 172.217.160.113:443: i/o timeout
      make: *** [install] Error 1
      

    • 如果需要設(shè)置代理 解決方法如下

      $ go env -w GOPROXY=https://goproxy.cn
      

• 三.部署goval-dictionary

  • 1.從github下載工具并安裝

    $ mkdir -p $GOPATH/src/github.com/kotakanbe
    $ cd $GOPATH/src/github.com/kotakanbe
    $ git clone https://github.com/kotakanbe/go-cve-dictionary.git
    $ cd go-cve-dictionary
    $ make install
    

  • 2.收集系統(tǒng)信息 （不同的系統(tǒng)命令不同 詳情參閱: goval fetch用法)

    $ goval-dictionary fetch-redhat 7
    

  • 踩坑

    • make install失敗3:還是包下載問題

      [root@izbp1268h8zj95audt9fdez goval-dictionary]# make install
      GO111MODULE=off go get -u golang.org/x/lint/golint
      package golang.org/x/lint/golint: unrecognized import path "golang.org/x/lint/golint": https fetch: Get "https://golang.org/x/lint/golint?go-get=1": dial tcp 216.239.37.1:443: i/o timeout
      make: *** [lint] Error 1
      

    • 一開始我以為還是代理的問題，設(shè)置來設(shè)置去還是不能解決尸饺，go env命令可以查看go的環(huán)境變量进统，我發(fā)現(xiàn)GO111MODULE已經(jīng)設(shè)置為on，但是執(zhí)行make install的時候侵佃，還是off麻昼，GO111MODULE涉及到包安裝的方式。我分析了安裝腳本之后發(fā)現(xiàn)腳本內(nèi)GO111MODULE設(shè)置了off馋辈，于是修改了腳本抚芦。

          [root@izbp1268h8zj95audt9fdez goval-dictionary]# cat GNUmakefile 
          .PHONY: \
              build \
              install \
              all \
              vendor \
              lint \
              vet \
              fmt \
              fmtcheck \
              pretest \
              test \
              cov \
              clean
      
          SRCS = $(shell git ls-files '*.go')
          PKGS =  ./commands ./config ./db ./db/rdb ./fetcher ./models ./util
          VERSION := $(shell git describe --tags --abbrev=0)
          REVISION := $(shell git rev-parse --short HEAD)
          LDFLAGS := -X 'main.version=$(VERSION)' \
              -X 'main.revision=$(REVISION)'
          GO := GO111MODULE=on go
          GO_OFF := GO111MODULE=off go
      
          all: build
      
          build: main.go pretest
              $(GO) build -a -ldflags "$(LDFLAGS)" -o goval-dictionary $<
      
          b:  main.go pretest
              $(GO) build -ldflags "$(LDFLAGS)" -o goval-dictionary $<
      
          install: main.go pretest
              $(GO) install -ldflags "$(LDFLAGS)"
      
          lint:
              $(GO_OFF) get -u golang.org/x/lint/golint      // 把lint方法中的GO_OFF改為GO 
              golint $(PKGS)
      
          vet:
              echo $(PKGS) | xargs env $(GO) vet || exit;
              ...
              ...
              ...
          ```
      

  • 踩坑

    • make install失敗4: sum.go missing 問題

      [root@izbp1268h8zj95audt9fdez goval-dictionary]# make install
      GO111MODULE=on go get -u golang.org/x/lint/golint
      golint ./commands ./config ./db ./db/rdb ./fetcher ./models ./util
      echo ./commands ./config ./db ./db/rdb ./fetcher ./models ./util | xargs env GO111MODULE=on go vet || exit;
      /root/go/pkg/mod/github.com/mattn/go-isatty@v0.0.12/isatty_tcgets.go:6:8: missing go.sum entry for module providing package golang.org/x/sys/unix (imported by github.com/mattn/go-isatty); to add:
          go get github.com/mattn/go-isatty@v0.0.12
      make: *** [vet] Error 123
      

    • go.sum 是一個校驗包的文件，多次嘗試迈螟，找到了一個解決方法 go get

      ```
      [root@izbp1268h8zj95audt9fdez goval-dictionary]# go get
      go: downloading golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4
      go: downloading github.com/dgrijalva/jwt-go v1.0.2
      go: downloading github.com/k0kubun/pp v1.3.0
      ```
      

• 四. 部署Vuls

  • 下載安裝工具

    $ mkdir -p $GOPATH/src/github.com/future-architect 
    $ cd $GOPATH/src/github.com/future-architect 
    $ git clone https://github.com/future-architect/vuls.git 
    $ cd vuls 
    $ make install
    

  • 踩坑

  • git 問題1: error: RPC failed; 這個問題出現(xiàn)了多次叉抡，可能是git需要下載的包太大，可能是防火墻或者代理等網(wǎng)絡(luò)情況

    [root@izbp1268h8zj95audt9fdez future-architect]# git clone https://github.com/future-architect/vuls.git
    Cloning into 'vuls'...
    remote: Enumerating objects: 7448, done.
    error: RPC failed; result=18, HTTP code = 200 | 20.00 KiB/s     
    fatal: The remote end hung up unexpectedly
    fatal: early EOF
    fatal: index-pack failed
    

  • 下面是我總結(jié)的解決方法 請逐一嘗試

    1. 更新git

    2. 檢查網(wǎng)絡(luò)答毫，瀏覽器訪問倉庫

    3. https:// 替換為 git://

    4. 增加緩沖區(qū)大小

       git config --global http.postBuffer 524288000
       

    5. 只拉去倉庫淺層目錄褥民，然后再拉取完整倉庫

       $ git clone http://github.com/large-repository --depth 1 
       $ cd large-repository 
       $ git fetch --unshallow
       

    • 網(wǎng)絡(luò)問題比較復(fù)雜，如果以上都不能起效洗搂，最后還可以手動下載復(fù)制git倉庫

    • 舉例

      ```
      $ mkdir -p $GOPATH/src/golang.org/x 
      $ git clone https://github.com/golang/lint.git $GOPATH/src/golang.org/x/lint 
      $ go get -u golang.org/x/lint/golint
      ```
      

  • 但是我的情況還是有錯 如下消返，最終修改Makefile文件 設(shè)置GO111MODULE=on 和上面提的一樣

      ```
      [root@izbp1268h8zj95audt9fdez vuls]# make install
      GO111MODULE=off go get -u golang.org/x/lint/golint
      package golang.org/x/tools/go/ast/astutil: unrecognized import path "golang.org/x/tools/go/ast/astutil": https fetch: Get "https://golang.org/x/tools/go/ast/astutil?go-get=1": dial tcp 216.239.37.1:443: i/o timeout
      ```
    

• 五.開始掃描和報告

  • 本地模式

  • 1.添加配置文件 .toml 文件內(nèi)容如下

    $ cd $HOME
    $ touch config.toml
    $ cat config.toml
    [servers]
    
    [servers.localhost]
    host = "localhost"
    port = "local"
    

  • 2.配置文件檢驗

    $ vuls configtest
    

  • 踩坑

    • 配置文件讀取失敗

      [root@izbp1268h8zj95audt9fdez ~]# vuls configtest
      [Mar  2 16:03:43]  INFO [localhost] vuls-v0.15.9-build-20210302_155557_1d0c5de
      [Mar  2 16:03:43]  INFO [localhost] Validating config...
      [Mar  2 16:03:43]  INFO [localhost] Detecting Server/Container OS... 
      [Mar  2 16:03:43]  INFO [localhost] Detecting OS of servers... 
      [Mar  2 16:03:43] ERROR [localhost] Failed to configtest: Failed to init servers. err:
          github.com/future-architect/vuls/scanner.Scanner.Configtest
              /root/go/src/github.com/future-architect/vuls/scanner/serverapi.go:107
        - No scannable host OS:
          github.com/future-architect/vuls/scanner.Scanner.initServers
              /root/go/src/github.com/future-architect/vuls/scanner/serverapi.go:235
      

    • 已聯(lián)系作者修改，更新至 ee38a87 提交以后 然后重新執(zhí)行第四步的make install 重新安裝

  • 3.開始掃描

    $ vuls scan
    ...
    ...
    ...
    Scan Summary
    ================
    localhost   centos7.7.1908  540 installed, 218 updatable   // 正確掃描結(jié)果
    

  • 踩坑

    • err: Failed to scan updatable packages

      [root@izbp1268h8zj95audt9fdez ~]# vuls scan
      [Mar  3 11:59:53]  INFO [localhost] vuls-v0.15.9-build-20210303_115852_1ec31d7
      [Mar  3 11:59:53]  INFO [localhost] Start scanning
      [Mar  3 11:59:53]  INFO [localhost] config: /root/config.toml
      [Mar  3 11:59:53]  INFO [localhost] Validating config...
      [Mar  3 11:59:53]  INFO [localhost] Detecting Server/Container OS... 
      [Mar  3 11:59:53]  INFO [localhost] Detecting OS of servers... 
      [Mar  3 11:59:53]  INFO [localhost] (1/1) Detected: localhost: centos 7.7.1908
      [Mar  3 11:59:53]  INFO [localhost] Detecting OS of containers... 
      [Mar  3 11:59:53]  INFO [localhost] Checking Scan Modes... 
      [Mar  3 11:59:53]  INFO [localhost] Detecting Platforms... 
      [Mar  3 11:59:55]  INFO [localhost] (1/1) localhost is running on other
      [Mar  3 11:59:55]  INFO [localhost] Detecting IPS identifiers... 
      [Mar  3 11:59:55]  INFO [localhost] (1/1) localhost has 0 IPS integration
      [Mar  3 11:59:55]  INFO [localhost] Scanning OS pkg in fast mode
      [Mar  3 12:00:05]  WARN [localhost] err: Failed to scan updatable packages:
          github.com/future-architect/vuls/scanner.(*redhatBase).scanPackages
              /root/go/src/github.com/future-architect/vuls/scanner/redhatbase.go:231
        - Failed to SSH: execResult: servername: 
            cmd: repoquery --all --pkgnarrow=updates --qf='%{NAME} %{EPOCH} %{VERSION} %{RELEASE} %{REPO}'
            exitstatus: 127
            stdout: 
            stderr: /bin/sh: repoquery: command not found
          
            err: exit status 127:
          github.com/future-architect/vuls/scanner.(*redhatBase).scanUpdatablePackages
              /root/go/src/github.com/future-architect/vuls/scanner/redhatbase.go:378
              
      

    • 解決辦法

      $ yum -y install yum-utils
      

  • 4.報告

    • 報告位置 /root/results/

    • View one-line summary

      $ vuls report -format-one-line-text
      

    • View short summary

      $ vuls report -format-list
      

    • View full report.

      $ vuls report -format-full-text | less
      

  • 5.進階報告查看

    • 終端展示

      $ vuls tui
      

      1.png

  • web UI 展示

    1. 獲得掃描數(shù)據(jù)

       // 需要現(xiàn)有掃描數(shù)據(jù) - 存放目錄 /root/results
       $ vuls scan
       $ vuls report -format-json
       

    2. 安裝WEB部署工具

       $ cd $HOME
       $ git clone https://github.com/ishiDACo/vulsrepo.git
       $ cd $HOME/vulsrepo/server
       $ cp vulsrepo-config.toml.sample vulsrepo-config.toml // 復(fù)制sample配置文件
       $ vi vulsrepo-config.toml                             // 根據(jù)自己的環(huán)境配置
       [Server]
       rootPath = "/home/vuls-user/vulsrepo"
       resultsPath  = "/opt/vuls/results"
       serverPort  = "5111"
       

    3. 運行

       $ pwd
       $HOME/vulsrepo/server
       
       $ ./vulsrepo-server
       

       I2ge.png

    WEB Demo :https://monocosel.jp/vulsrepo/