Container中運(yùn)行Windows虛擬機(jī)

[toc]

Windows in container

1 意義

可基于一個image run很多的container镶殷，每一個container都是獨(dú)立的。所占用的存儲空間僅僅是一個image的大小。并且基于容器技術(shù)冬三，更方便遷移

2 架構(gòu)

在Linux container（此處選用Ubuntu18.04）中安裝kernel image琳状，再安裝kvm來運(yùn)行虛擬機(jī)


image.png

3 前提

確保你的宿主機(jī)能夠支持虛擬化

3.1 確認(rèn)步驟

輸入以下命令，如果輸出大于0膘螟，則證明你的系統(tǒng)支持kvm虛擬化，如果輸出等于0碾局，則需要在BIOS中開啟硬件虛擬化

sudo egrep -c '(vmx|svm)' /proc/cpuinfo

3.2 VMware虛擬機(jī)開啟硬件虛擬化

VMware Workstations開啟虛擬機(jī)虛擬化


image.png

VMware vCenter開啟虛擬機(jī)虛擬化


image.png

4 步驟

4.1 安裝docker

4.2 運(yùn)行一個Ubuntu 18.04的容器

命令

docker run --privileged -p 3389:3389 -itd --name ubuntukvm --device=/dev/kvm --device=/dev/net/tun -v /sys/fs/cgroup:/sys/fs/cgroup:rw --cap-add=NET_ADMIN --cap-add=SYS_ADMIN ubuntu:18.04 /bin/bash

參數(shù)含義

--privileged：使用該參數(shù)能讓container內(nèi)的root擁有真正的root權(quán)限
--itd：it讓容器開啟一個偽終端荆残，d讓容器能夠后臺運(yùn)行，避免出現(xiàn)退出容器終端后容器stop
--device：添加一個宿主機(jī)的設(shè)備到容器
-v：掛載目錄
--cap-add：Add Linux capabilities

4.3 進(jìn)入容器交互

命令

docker exec -it ubuntukvm /bin/bash

4.3.1 安裝kvm工具

命令

apt-get update -y
apt-get install -y qemu-kvm libvirt-daemon-system libvirt-dev

4.3.2 服務(wù)啟動

命令

chown root:kvm /dev/kvm
service libvirtd start
service virtlogd start

4.3.3 安裝kernel image

命令

apt-get install -y linux-image-$(uname -r)

由于我的宿主機(jī)也是Ubuntu18.04净当，所以可以使用$(uname -r)内斯，如果宿主機(jī)不是Ubuntu 18.04，需要去查找適合的kernel image版本

4.3.4 安裝Vagrant

命令

apt-get install curl -y
apt-get install net-tools -y
apt-get install jq -y
vagrant_latest_version=$(curl -s https://checkpoint-api.hashicorp.com/v1/check/vagrant  | jq -r -M '.current_version')
echo $vagrant_latest_version
curl -O https://releases.hashicorp.com/vagrant/$(echo $vagrant_latest_version)/vagrant_$(echo $vagrant_latest_version)_x86_64.deb
dpkg -i vagrant_$(echo $vagrant_latest_version)_x86_64.deb
vagrant plugin install vagrant-libvirt

4.3.5 下載Windows 虛擬機(jī)Box

命令

mkdir /win10
cd /win10
vagrant init peru/windows-server-2012_r2-standard-x64-eval

4.3.6 啟動虛擬機(jī)

命令及結(jié)果

root@15421e73344e:/win# VAGRANT_DEFAULT_PROVIDER=libvirt vagrant up
Bringing machine 'default' up with 'libvirt' provider...
==> default: Checking if box 'peru/windows-server-2012_r2-standard-x64-eval' version '20210104.01' is up to date...
==> default: Starting domain.
==> default: Error when updating domain settings: undefined method `strip' for nil:NilClass
==> default: Waiting for domain to get an IP address...
==> default: Waiting for SSH to become available...
==> default: Creating shared folders metadata...
==> default: Forwarding ports...
==> default: 3389 (guest) => 3389 (host) (adapter eth0)
==> default: 5986 (guest) => 5986 (host) (adapter eth0)
==> default: 5985 (guest) => 5985 (host) (adapter eth0)

4.3.7 網(wǎng)絡(luò)配置

命令及結(jié)果

root@15421e73344e:/win# vagrant rdp
==> default: Detecting RDP info...
    default: Address: 192.168.121.46:3389
    default: Username: vagrant
==> default: Vagrant will now launch your RDP client with the connection parameters
==> default: above. If the connection fails, verify that the information above is
==> default: correct. Additionally, make sure the RDP server is configured and
==> default: running in the guest machine (it is disabled by default on Windows).
==> default: Also, verify that the firewall is open to allow RDP connections.
An appropriate RDP client was not found. Vagrant requires either
`xfreerdp` or `rdesktop` in order to connect via RDP to the Vagrant
environment. Please ensure one of these applications is installed and
available on the path and try again.

命令

iptables-save > $HOME/firewall.txt
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT

iptables -A FORWARD -i eth0 -o virbr1 -p tcp --syn --dport 3389 -m conntrack --ctstate NEW -j ACCEPT
iptables -A FORWARD -i eth0 -o virbr1 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -i virbr1 -o eth0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 3389 -j DNAT --to-destination 192.168.121.68
iptables -t nat -A POSTROUTING -o virbr1 -p tcp --dport 3389 -d 192.168.121.68 -j SNAT --to-source 192.168.121.1

iptables -D FORWARD -o virbr1 -j REJECT --reject-with icmp-port-unreachable
iptables -D FORWARD -i virbr1 -j REJECT --reject-with icmp-port-unreachable
iptables -D FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable
iptables -D FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable

4.4 通過RDP連接虛擬機(jī)

4.4.1 局域網(wǎng)Windows連接

打開遠(yuǎn)程桌面連接


image.png

輸入ip地址以及端口號進(jìn)行連接


image.png

賬號密碼都為vagrant


image.png

image.png

5 創(chuàng)建docker image

5.1 commit 成新image

命令及結(jié)果

root@ubuntu:/# docker commit 15421e73344e  teym88/ubuntukvm
sha256:1e2f2b486882d835e77f790049ed7b805bae7871ef712ff8bf3f702ba71b4903

5.2 創(chuàng)建DockerFile

內(nèi)容

FROM teym88/ubuntukvm
COPY startup.sh /

startup.sh內(nèi)容

#!/bin/bash
#set -eou pipefail

chmod 777 /dev/kvm
service libvirtd start
service virtlogd start
cd win/
VAGRANT_DEFAULT_PROVIDER=libvirt vagrant up
vagrant rdp
iptables-save > $HOME/firewall.txt
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT

iptables -A FORWARD -i eth0 -o virbr1 -p tcp --syn --dport 3389 -m conntrack --ctstate NEW -j ACCEPT
iptables -A FORWARD -i eth0 -o virbr1 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -i virbr1 -o eth0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 3389 -j DNAT --to-destination 192.168.121.46
iptables -t nat -A POSTROUTING -o virbr1 -p tcp --dport 3389 -d 192.168.121.46 -j SNAT --to-source 192.168.121.1

iptables -D FORWARD -o virbr1 -j REJECT --reject-with icmp-port-unreachable
iptables -D FORWARD -i virbr1 -j REJECT --reject-with icmp-port-unreachable
iptables -D FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable
iptables -D FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable

5.3 Build 新image

命令及結(jié)果

root@ubuntu:~/dockerwin# docker build -f Dockerfile --tag teym88/ubuntuwinkvm . 
Sending build context to Docker daemon  4.096kB
Step 1/2 : FROM teym88/ubuntukvm
 ---> 1e2f2b486882
Step 2/2 : COPY startup.sh /
 ---> d847dd970e49
Successfully built d847dd970e49
Successfully tagged teym88/ubuntuwinkvm:latest

5.4 使用新image運(yùn)行

命令

root@ubuntu:~/dockerwin# docker run --privileged -itd --name kvmcontainer --device=/dev/kvm --device=/dev/net/tun -v /sys/fs/cgroup:/sys/fs/cgroup:rw --cap-add=NET_ADMIN --cap-add=SYS_ADMIN -p 3396:3389 teym88/ubuntuwinkvm /bin/bash
6abd67eb0c8f3e3202c04ad6179819712dfb666952b7f7115cf6637c88c8c439

5.5 到容器內(nèi)部啟動虛擬機(jī)

通過運(yùn)行startup.sh即可

root@ubuntu:~/dockerwin# docker exec -it kvmcontainer /bin/bash
root@6abd67eb0c8f:/# ls
bin   dev  home        initrd.img.old  lib64  mnt  proc  run   srv         sys  usr                        var      vmlinuz.old
boot  etc  initrd.img  lib             media  opt  root  sbin  startup.sh  tmp  vagrant_2.2.14_x86_64.deb  vmlinuz  win
root@6abd67eb0c8f:/# st  
start-stop-daemon  stat               stdbuf             strings            strip              stty               
root@6abd67eb0c8f:/# /s                
sbin/       srv/        startup.sh  sys/        
root@6abd67eb0c8f:/# /startup.sh 
 * Starting libvirt management daemon libvirtd                                                                                                                             [ OK ] 
 * Starting libvirt logging daemon virtlogd                                                                                                                                [ OK ] 
Bringing machine 'default' up with 'libvirt' provider...
==> default: Checking if box 'peru/windows-server-2012_r2-standard-x64-eval' version '20210104.01' is up to date...
==> default: Starting domain.
==> default: Error when updating domain settings: undefined method `strip' for nil:NilClass
==> default: Waiting for domain to get an IP address...
==> default: Waiting for SSH to become available...
==> default: Creating shared folders metadata...
==> default: Forwarding ports...
==> default: 3389 (guest) => 3389 (host) (adapter eth0)
==> default: 5986 (guest) => 5986 (host) (adapter eth0)
==> default: 5985 (guest) => 5985 (host) (adapter eth0)
==> default: Machine already provisioned. Run `vagrant provision` or use the `--provision`
==> default: flag to force provisioning. Provisioners marked to run always will still run.
==> default: Detecting RDP info...
    default: Address: 192.168.121.46:3389
    default: Username: vagrant
==> default: Vagrant will now launch your RDP client with the connection parameters
==> default: above. If the connection fails, verify that the information above is
==> default: correct. Additionally, make sure the RDP server is configured and
==> default: running in the guest machine (it is disabled by default on Windows).
==> default: Also, verify that the firewall is open to allow RDP connections.
An appropriate RDP client was not found. Vagrant requires either
`xfreerdp` or `rdesktop` in order to connect via RDP to the Vagrant
environment. Please ensure one of these applications is installed and
available on the path and try again.

擴(kuò)展

給虛擬機(jī)加盤

需要通過kvm方式進(jìn)行添加

創(chuàng)建Block 類型的persistent volume

添加到Pod

yaml文件

將lstblk和blk10這兩個pvc掛載到VmContainer的/dev/sdh和/dev/sdi

apiVersion: v1
kind: Pod
metadata:
  name: winvm
  labels: 
    app: containerVm
spec:
  nodeName: k8smaster
  containers:
  - name: win
    image: teym88/ubuntukvmwin
    command: ["/bin/bash", "-ce", "tail -f /dev/null"]
    securityContext:
      privileged : true
      capabilities:
        add: 
        - NET_ADMIN
        - SYS_ADMIN
    ports:
    - containerPort: 3389
    volumeMounts:
    - mountPath: /sys/fs/cgroup
      name: cgroup
    - mountPath: /sys
      name: sys
    volumeDevices:
    - name: data
      devicePath: /dev/sdh
    - name: blk10
      devicePath: /dev/sdi
  volumes:
  - name: cgroup
    hostPath:
      path: /sys/fs/cgroup
      type: Directory
  - name: sys
    hostPath:
      path: /sys
      type: Directory
  - name: data
    persistentVolumeClaim:
      claimName: lstblk
  - name: blk10
    persistentVolumeClaim:
      claimName: blk10

注意

無法在線給Pod添加pvc像啼，如果是使用pod來運(yùn)行VmContainer俘闯，只能在一開始就添加pvc

The Pod "winvm" is invalid: spec: Forbidden: pod updates may not change fields other than `spec.containers[*].image`, `spec.initContainers[*].image`, `spec.activeDeadlineSeconds` or `spec.tolerations` (only additions to existing tolerations)

嘗試在vagrant up之前給虛擬機(jī)加盤

使用virsh編輯虛擬機(jī)xml文件

由于無法使用vagrant的方式來給虛擬機(jī)加盤，所以使用Libvirt的管理軟件virsh來進(jìn)行添加

查找虛擬機(jī)名稱

進(jìn)入pod 交互

root@k8smaster:~# kubectl exec -it winvm -- /bin/bash
root@winvm:/# ls

查看添加的硬盤

root@winvm:/# ls -l /dev |grep sd
brw-rw---- 1 root disk      8,    0 Mar 11 02:02 sda
brw-rw---- 1 root disk      8,    1 Mar 11 02:02 sda1
brw-rw---- 1 root disk      8,   16 Mar 11 02:02 sdb
brw-rw---- 1 root disk    147, 1003 Mar 11 02:02 sdh
brw-rw---- 1 root disk    147, 1004 Mar 11 02:02 sdi

查看虛擬機(jī)

root@winvm:/# virsh list --all      
 Id    Name                           State
----------------------------------------------------
 -     win_default                    shut off
編輯xml文件

使用virsh edit win_default命令進(jìn)入虛擬機(jī)配置文件編輯模式忽冻，添加<disk>字段

<disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source dev='/dev/sdi'/>
      <target dev='vdb' bus='virtio'/>
</disk>
使用vagrant 命令開啟虛擬機(jī)
root@winvm:/win# VAGRANT_DEFAULT_PROVIDER=libvirt vagrant up
Bringing machine 'default' up with 'libvirt' provider...
==> default: Checking if box 'peru/windows-server-2012_r2-standard-x64-eval' version '20210104.01' is up to date...
==> default: Starting domain.
==> default: Error when updating domain settings: undefined method `strip' for nil:NilClass
==> default: Waiting for domain to get an IP address...
==> default: Waiting for SSH to become available...
==> default: Creating shared folders metadata...
==> default: Forwarding ports...
==> default: 3389 (guest) => 3389 (host) (adapter eth0)
==> default: 5986 (guest) => 5986 (host) (adapter eth0)
==> default: 5985 (guest) => 5985 (host) (adapter eth0)
==> default: Machine already provisioned. Run `vagrant provision` or use the `--provision`
==> default: flag to force provisioning. Provisioners marked to run always will still run.

遠(yuǎn)程連接虛擬機(jī)

可以看到已經(jīng)增加一個disk


image.png

嘗試在虛擬機(jī)運(yùn)行之后加盤

如上繼續(xù)使用virsh編輯xml文件

使用vagrant reload 虛擬機(jī)

報錯真朗，無法重啟

==> default: Clearing any previously set forwarded ports...
==> default: Halting domain...
Traceback (most recent call last):
        62: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/bin/vagrant:205:in `<main>'
        61: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/environment.rb:290:in `cli'
        60: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/cli.rb:67:in `execute'
        59: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/plugins/commands/reload/command.rb:40:in `execute'
        58: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/plugin/v2/command.rb:232:in `with_target_vms'
        57: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/plugin/v2/command.rb:232:in `each'
        56: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/plugin/v2/command.rb:243:in `block in with_target_vms'
        55: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/plugins/commands/reload/command.rb:42:in `block in execute'
        54: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/machine.rb:201:in `action'
        53: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/machine.rb:201:in `call'
        52: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/environment.rb:613:in `lock'
        51: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/machine.rb:215:in `block in action'
        50: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/machine.rb:246:in `action_raw'
        49: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/runner.rb:89:in `run'
        48: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/util/busy.rb:19:in `busy'
        47: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/runner.rb:89:in `block in run'
        46: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builder.rb:149:in `call'
        45: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        44: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builtin/call.rb:53:in `call'
        43: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/runner.rb:89:in `run'
        42: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/util/busy.rb:19:in `busy'
        41: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/runner.rb:89:in `block in run'
        40: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builder.rb:149:in `call'
        39: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        38: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:127:in `block in finalize_action'
        37: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        36: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builtin/config_validate.rb:25:in `call'
        35: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        34: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builtin/config_validate.rb:25:in `call'
        33: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        32: from /root/.vagrant.d/gems/2.6.6/gems/vagrant-libvirt-0.3.0/lib/vagrant-libvirt/action/forward_ports.rb:199:in `call'
        31: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        30: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builtin/call.rb:53:in `call'
        29: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/runner.rb:89:in `run'
        28: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/util/busy.rb:19:in `busy'
        27: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/runner.rb:89:in `block in run'
        26: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builder.rb:149:in `call'
        25: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        24: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:127:in `block in finalize_action'
        23: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        22: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builtin/call.rb:53:in `call'
        21: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/runner.rb:89:in `run'
        20: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/util/busy.rb:19:in `busy'
        19: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/runner.rb:89:in `block in run'
        18: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builder.rb:149:in `call'
        17: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        16: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:127:in `block in finalize_action'
        15: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        14: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:127:in `block in finalize_action'
        13: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        12: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builtin/call.rb:53:in `call'
        11: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/runner.rb:89:in `run'
        10: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/util/busy.rb:19:in `busy'
         9: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/runner.rb:89:in `block in run'
         8: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builder.rb:149:in `call'
         7: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
         6: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:127:in `block in finalize_action'
         5: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
         4: from /root/.vagrant.d/gems/2.6.6/gems/vagrant-libvirt-0.3.0/lib/vagrant-libvirt/action/halt_domain.rb:28:in `call'
         3: from /root/.vagrant.d/gems/2.6.6/gems/fog-core-2.2.3/lib/fog/core/model.rb:72:in `wait_for'
         2: from /root/.vagrant.d/gems/2.6.6/gems/fog-core-2.2.3/lib/fog/core/wait_for.rb:6:in `wait_for'
         1: from /root/.vagrant.d/gems/2.6.6/gems/fog-core-2.2.3/lib/fog/core/wait_for.rb:6:in `loop'
/root/.vagrant.d/gems/2.6.6/gems/fog-core-2.2.3/lib/fog/core/wait_for.rb:9:in `block in wait_for': The specified wait_for timeout (30 seconds) was exceeded (Fog::Errors::TimeoutError)
        62: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/bin/vagrant:205:in `<main>'
        61: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/environment.rb:290:in `cli'
        60: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/cli.rb:67:in `execute'
        59: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/plugins/commands/reload/command.rb:40:in `execute'
        58: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/plugin/v2/command.rb:232:in `with_target_vms'
        57: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/plugin/v2/command.rb:232:in `each'
        56: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/plugin/v2/command.rb:243:in `block in with_target_vms'
        55: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/plugins/commands/reload/command.rb:42:in `block in execute'
        54: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/machine.rb:201:in `action'
        53: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/machine.rb:201:in `call'
        52: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/environment.rb:613:in `lock'
        51: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/machine.rb:215:in `block in action'
        50: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/machine.rb:246:in `action_raw'
        49: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/runner.rb:89:in `run'
        48: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/util/busy.rb:19:in `busy'
        47: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/runner.rb:89:in `block in run'
        46: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builder.rb:149:in `call'
        45: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        44: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builtin/call.rb:53:in `call'
        43: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/runner.rb:89:in `run'
        42: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/util/busy.rb:19:in `busy'
        41: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/runner.rb:89:in `block in run'
        40: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builder.rb:149:in `call'
        39: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        38: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:127:in `block in finalize_action'
        37: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        36: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builtin/config_validate.rb:25:in `call'
        35: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        34: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builtin/config_validate.rb:25:in `call'
        33: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        32: from /root/.vagrant.d/gems/2.6.6/gems/vagrant-libvirt-0.3.0/lib/vagrant-libvirt/action/forward_ports.rb:199:in `call'
        31: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        30: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builtin/call.rb:53:in `call'
        29: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/runner.rb:89:in `run'
        28: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/util/busy.rb:19:in `busy'
        27: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/runner.rb:89:in `block in run'
        26: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builder.rb:149:in `call'
        25: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        24: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:127:in `block in finalize_action'
        23: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        22: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builtin/call.rb:53:in `call'
        21: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/runner.rb:89:in `run'
        20: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/util/busy.rb:19:in `busy'
        19: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/runner.rb:89:in `block in run'
        18: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builder.rb:149:in `call'
        17: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        16: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:127:in `block in finalize_action'
        15: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        14: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:127:in `block in finalize_action'
        13: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
        12: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builtin/call.rb:53:in `call'
        11: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/runner.rb:89:in `run'
        10: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/util/busy.rb:19:in `busy'
         9: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/runner.rb:89:in `block in run'
         8: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/builder.rb:149:in `call'
         7: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
         6: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:127:in `block in finalize_action'
         5: from /opt/vagrant/embedded/gems/2.2.14/gems/vagrant-2.2.14/lib/vagrant/action/warden.rb:48:in `call'
         4: from /root/.vagrant.d/gems/2.6.6/gems/vagrant-libvirt-0.3.0/lib/vagrant-libvirt/action/halt_domain.rb:27:in `call'
         3: from /root/.vagrant.d/gems/2.6.6/gems/vagrant-libvirt-0.3.0/lib/vagrant-libvirt/action/halt_domain.rb:33:in `rescue in call'
         2: from /root/.vagrant.d/gems/2.6.6/gems/fog-libvirt-0.8.0/lib/fog/libvirt/models/compute/server.rb:107:in `poweroff'
         1: from /root/.vagrant.d/gems/2.6.6/gems/fog-libvirt-0.8.0/lib/fog/libvirt/requests/compute/vm_action.rb:7:in `vm_action'
/root/.vagrant.d/gems/2.6.6/gems/fog-libvirt-0.8.0/lib/fog/libvirt/requests/compute/vm_action.rb:7:in `destroy': Call to virDomainDestroyFlags failed: Failed to terminate process 274 with SIGKILL: Device or resource busy (Libvirt::Error)

使用virsh重啟虛擬機(jī)

報錯，一直處于shutting down的狀態(tài)

root@winvm:/win# virsh destroy win_default
error: Disconnected from qemu:///system due to keepalive timeout
error: Failed to destroy domain win_default
error: internal error: connection closed due to keepalive timeout

root@winvm:/win# virsh list
 Id    Name                           State
----------------------------------------------------
 4     win_default                    in shutdown

root@winvm:/win# exit
最后編輯于 ：
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者
  • 人面猴
    序言：七十年代末甚颂，一起剝皮案震驚了整個濱河市蜜猾，隨后出現(xiàn)的幾起案子，更是在濱河造成了極大的恐慌振诬，老刑警劉巖蹭睡，帶你破解...
    沈念sama閱讀 218,386評論 6贊 506
  • 死咒
    序言：濱河連續(xù)發(fā)生了三起死亡事件，死亡現(xiàn)場離奇詭異赶么，居然都是意外死亡肩豁，警方通過查閱死者的電腦和手機(jī)，發(fā)現(xiàn)死者居然都...
    沈念sama閱讀 93,142評論 3贊 394
  • 救了他兩次的神仙讓他今天三更去死
    文/潘曉璐 我一進(jìn)店門辫呻，熙熙樓的掌柜王于貴愁眉苦臉地迎上來清钥，“玉大人，你說我怎么就攤上這事放闺∷钫眩” “怎么了？”我有些...
    開封第一講書人閱讀 164,704評論 0贊 353
  • 道士緝兇錄：失蹤的賣姜人
    文/不壞的土叔 我叫張陵怖侦，是天一觀的道長篡悟。 經(jīng)常有香客問我谜叹，道長，這世上最難降的妖魔是什么搬葬？ 我笑而不...
    開封第一講書人閱讀 58,702評論 1贊 294
  • ?港島之戀（遺憾婚禮）
    正文 為了忘掉前任荷腊，我火速辦了婚禮，結(jié)果婚禮上急凰，老公的妹妹穿的比我還像新娘女仰。我一直安慰自己，他們只是感情好抡锈，可當(dāng)我...
    茶點(diǎn)故事閱讀 67,716評論 6贊 392
  • 惡毒庶女頂嫁案：這布局不是一般人想出來的
    文/花漫 我一把揭開白布疾忍。 她就那樣靜靜地躺著，像睡著了一般企孩。 火紅的嫁衣襯著肌膚如雪锭碳。 梳的紋絲不亂的頭發(fā)上，一...
    開封第一講書人閱讀 51,573評論 1贊 305
  • 城市分裂傳說
    那天勿璃，我揣著相機(jī)與錄音，去河邊找鬼推汽。 笑死补疑，一個胖子當(dāng)著我的面吹牛，可吹牛的內(nèi)容都是我干的歹撒。 我是一名探鬼主播莲组，決...
    沈念sama閱讀 40,314評論 3贊 418
  • 雙鴛鴦連環(huán)套：你想象不到人心有多黑
    文/蒼蘭香墨 我猛地睜開眼，長吁一口氣：“原來是場噩夢啊……” “哼暖夭！你這毒婦竟也來了锹杈？” 一聲冷哼從身側(cè)響起，我...
    開封第一講書人閱讀 39,230評論 0贊 276
  • 萬榮殺人案實(shí)錄
    序言：老撾萬榮一對情侶失蹤迈着，失蹤者是張志新（化名）和其女友劉穎竭望，沒想到半個月后，有當(dāng)?shù)厝嗽跇淞掷锇l(fā)現(xiàn)了一具尸體裕菠，經(jīng)...
    沈念sama閱讀 45,680評論 1贊 314
  • ?護(hù)林員之死
    正文 獨(dú)居荒郊野嶺守林人離奇死亡咬清，尸身上長有42處帶血的膿包…… 初始之章·張勛 以下內(nèi)容為張勛視角 年9月15日...
    茶點(diǎn)故事閱讀 37,873評論 3贊 336
  • ?白月光啟示錄
    正文 我和宋清朗相戀三年，在試婚紗的時候發(fā)現(xiàn)自己被綠了奴潘。 大學(xué)時的朋友給我發(fā)了我未婚夫和他白月光在一起吃飯的照片旧烧。...
    茶點(diǎn)故事閱讀 39,991評論 1贊 348
  • 活死人
    序言：一個原本活蹦亂跳的男人離奇死亡，死狀恐怖画髓，靈堂內(nèi)的尸體忽然破棺而出掘剪，到底是詐尸還是另有隱情，我是刑警寧澤奈虾，帶...
    沈念sama閱讀 35,706評論 5贊 346
  • ?日本核電站爆炸內(nèi)幕
    正文 年R本政府宣布夺谁，位于F島的核電站肆汹，受9級特大地震影響，放射性物質(zhì)發(fā)生泄漏予权。R本人自食惡果不足惜昂勉，卻給世界環(huán)境...
    茶點(diǎn)故事閱讀 41,329評論 3贊 330
  • 男人毒藥：我在死后第九天來索命
    文/蒙蒙 一、第九天 我趴在偏房一處隱蔽的房頂上張望扫腺。 院中可真熱鬧岗照，春花似錦、人聲如沸笆环。這莊子的主人今日做“春日...
    開封第一講書人閱讀 31,910評論 0贊 22
  • 一樁弒父案，背后竟有這般陰謀
    文/蒼蘭香墨 我抬頭看了看天上的太陽躁劣。三九已至迫吐，卻和暖如春，著一層夾襖步出監(jiān)牢的瞬間账忘，已是汗流浹背志膀。 一陣腳步聲響...
    開封第一講書人閱讀 33,038評論 1贊 270
  • 情欲美人皮
    我被黑心中介騙來泰國打工， 沒想到剛下飛機(jī)就差點(diǎn)兒被人妖公主榨干…… 1. 我叫王不留鳖擒，地道東北人溉浙。 一個月前我還...
    沈念sama閱讀 48,158評論 3贊 370
  • 代替公主和親
    正文 我出身青樓，卻偏偏與公主長得像蒋荚，于是被迫代替她去往敵國和親戳稽。 傳聞我的和親對象是個殘疾皇子，可洞房花燭夜當(dāng)晚...
    茶點(diǎn)故事閱讀 44,941評論 2贊 355

推薦閱讀更多精彩內(nèi)容