nova file injection的原理和調試過程

file injection代碼

file injection原理來說是比較簡單的，在nova boot命令中，有參數--file，是將文件inject到image中

nova boot --flavor 2 --image d96b0e41-8264-41de-8dbb-6b31ce9bfbfc --key-name openstack --security-groups default --file /home/ubuntu/bootfromvolume1.xml=/home/cliu8/images/bootfromvolume.xml testinject19

這裏容易混淆的是還有一個參數是user_data，這不是file injection，而是cloud-init使用這裏面的東西對系統進行初始化。

在虛擬機啓動的時候，會調用到/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py中的spawn函數

def spawn(self, context, instance, image_meta, injected_files,
          admin_password, network_info=None, block_device_info=None):
    disk_info = blockinfo.get_disk_info(CONF.libvirt.virt_type,
                                        instance,
                                        block_device_info,
                                        image_meta)
    self._create_image(context, instance,
                       disk_info['mapping'],
                       network_info=network_info,
                       block_device_info=block_device_info,
                       files=injected_files,
                       admin_pass=admin_password)
    xml = self.to_xml(context, instance, network_info,
                      disk_info, image_meta,
                      block_device_info=block_device_info,
                      write_to_disk=True)

    self._create_domain_and_network(context, xml, instance, network_info,
                                    block_device_info)
    LOG.debug(_("Instance is running"), instance=instance)

其中一步是create image，在_create_image函數中，有下面的邏輯

elif inject_files and CONF.libvirt.inject_partition != -2:
    if booted_from_volume:
        LOG.warn(_('File injection into a boot from volume '
                   'instance is not supported'), instance=instance)
    self._inject_data(
        instance, network_info, admin_pass, files, suffix)

當有文件要inject的時候，而且參數inject_partition不設爲-2，則執行file injection過程。

從這裏咱們能夠看出，在參數中應該添加

# cat /etc/nova/nova-compute.conf
[DEFAULT]
compute_driver=libvirt.LibvirtDriver
[libvirt]
virt_type=kvm
inject_partition=-1

在代碼中，咱們能夠看到這個參數是這樣解釋的：

cfg.IntOpt('inject_partition',
            default=-2,
            help='The partition to inject to : '
                 '-2 => disable, -1 => inspect (libguestfs only), '
                 '0 => not partitioned, >0 => partition number',
           deprecated_name='libvirt_inject_partition',
           deprecated_group='DEFAULT'),

在最新的代碼中，咱們每每選擇用ligguestfs來進行文件注入，於是設爲-1

在_inject_data函數中，

disk.inject_data(injection_path,
                 key, net, metadata, admin_pass, files,
                 partition=target_partition,
                 use_cow=CONF.use_cow_images,
                 mandatory=('files',))

(Pdb) p injection_path
'/var/lib/nova/instances/fedc59a9-6ff5-4c23-9595-d877a6796c84/disk'

(Pdb) p target_partition
-1

這裏key, net, metadata, admin_pass咱們都不注入，因此都爲空，只有files有值

這裏就調用到/usr/lib/python2.7/dist-packages/nova/virt/disk/api.py的

def inject_data(image, key=None, net=None, metadata=None, admin_password=None,
                files=None, partition=None, use_cow=False, mandatory=()):

在這個函數中首先生成一個對象來操做這個image

fs = vfs.VFS.instance_for_image(image, fmt, partition)

若是可以找到guestfs，則fs爲nova.virt.disk.vfs.guestfs.VFSGuestFS

接着調用來初始化環境，該run起來的guestfs的applicance要運行起來，該mount的image要mount上去

fs.setup()

最後進行注入操做

return inject_data_into_fs(fs, key, net, metadata,
                           admin_password, files, mandatory)

在/usr/lib/python2.7/dist-packages/nova/virt/disk/vfs/guestfs.py的setup函數中，

並非調用guestfish命令，而是直接經過python調用guestfsd的API，能夠參考http://libguestfs.org/guestfs.3.html

先要有個handle

self.handle = tpool.Proxy(guestfs.GuestFS(close_on_exit=False))

而後將image做爲一個drive添加到appliance

self.handle.add_drive_opts(self.imgfile, format=self.imgfmt)

運行appliance

self.handle.launch()

self.setup_os()

self.handle.aug_init("/", 0)

setup_os的實現很簡單，因爲咱們的參數是-1，則調用setup_os_inspect

def setup_os(self):
    if self.partition == -1:
        self.setup_os_inspect()
    else:
        self.setup_os_static()

在函數setup_os_inspect中

先看有多少個root disk

roots = self.handle.inspect_os()

(Pdb) p roots
['/dev/sda1']

發現只有一個而後調用

self.setup_os_root(roots[0])

在setup_os_root中，先是獲得mount point

mounts = self.handle.inspect_get_mountpoints(root)

(Pdb) p mounts
[('/', '/dev/sda1')]

最後self.handle.mount_options("", mount[1], mount[0])

(Pdb) p mount[1]
'/dev/sda1'
(Pdb) p mount[0]
'/'

將/dev/sda1掛載到/下面

至此appliance起來了，image也掛載了, fs.setup()成功，接下來是調用inject_data_into_fs

inject_data_into_fs對不一樣的注入對象調用不一樣的函數，對於文件注入調用def _inject_files_into_fs(files, fs):

這裏面是一個循環，每一個文件都調用def _inject_file_into_fs(fs, path, contents, append=False):

這裏面調用了fs.replace_file(path, contents)

實際上是寫一個文件到一個path

def replace_file(self, path, content):
    LOG.debug(_("Replace file path=%s"), path)
    path = self._canonicalize_path(path)
    self.handle.write(path, content)

(Pdb) p path
u'/home/ubuntu/bootfromvolume1.xml'
(Pdb) p content
"<domain type='kvm'>\n  <name>bootfromvolume</name>\n  <uuid>0f0806ab-531d-6134-5def-c5b495529289</uuid>\n  <memory unit='KiB'>2097152</memory>\n  <currentMemory unit='KiB'>2097152</currentMemory>\n  <vcpu placement='static'>1</vcpu>\n  <os>\n    <type arch='x86_64' machine='pc-i440fx-trusty'>hvm</type>\n    <boot dev='hd'/>\n  </os>\n  <features>\n    <acpi/>\n    <apic/>\n    <pae/>\n  </features>\n  <clock offset='utc'/>\n  <on_poweroff>destroy</on_poweroff>\n  <on_reboot>restart</on_reboot>\n  <on_crash>restart</on_crash>\n  <devices>\n    <emulator>/usr/bin/kvm-spice</emulator>\n    <disk type='block' device='disk'>\n      <driver name='qemu' type='raw' cache='none'/>\n      <source dev='/dev/disk/by-path/ip-16.158.166.197:3260-iscsi-iqn.2010-10.org.openstack:volume-cliu8-test-lun-1'/>\n      <target dev='vda' bus='virtio'/>\n      <serial>640a10f7-3965-4a47-9641-002a94526445</serial>\n      <alias name='virtio-disk0'/>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>\n    </disk>\n    <controller type='usb' index='0'>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>\n    </controller>\n    <controller type='pci' index='0' model='pci-root'/>\n    <controller type='ide' index='0'>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>\n    </controller>\n    <interface type='network'>\n      <mac address='52:54:00:9b:d5:cc'/>\n      <source network='default'/>\n      <model type='rtl8139'/>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>\n    </interface>    \n    <serial type='pty'>\n      <target port='0'/>\n    </serial>\n    <console type='pty'>\n      <target type='serial' port='0'/>\n    </console>\n    <input type='mouse' bus='ps2'/>\n    <input type='keyboard' bus='ps2'/>\n    <graphics type='vnc' port='-1' autoport='yes' listen='0.0.0.0'>\n      <listen type='address' address='0.0.0.0'/>\n    </graphics>\n    <video>\n      <model type='cirrus' vram='9216' heads='1'/>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>\n    </video>\n    <memballoon model='virtio'>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>\n    </memballoon>\n  </devices>\n</domain>\n"

至此文件注入完畢。

 

file injection調經過程

上面的過程，理論上很簡單，可是真正調通，仍是經歷了複雜的過程

個人環境是Ubuntu 14.04, apt-get安裝的Icehouse

首先是這個錯誤

2014-07-09 22:50:12.359 7885 WARNING nova.virt.disk.vfs.guestfs [req-b3655a75-1607-4c90-b7cb-cc81e543e769 5df0e89888364c0b80d
47a7a426a9a67 c24c59846a7f44538d958e7548cc74a3] Failed to close augeas aug_close: call launch before using this function
(in guestfish, don't forget to use the 'run' command)
2014-07-09 22:50:12.366 7885 ERROR nova.virt.libvirt.driver [req-b3655a75-1607-4c90-b7cb-cc81e543e769 5df0e89888364c0b80d47a7
a426a9a67 c24c59846a7f44538d958e7548cc74a3] [instance: 7bf0f046-69a7-4c5f-990b-9cbfaf3b0b55] Error injecting data into image
d96b0e41-8264-41de-8dbb-6b31ce9bfbfc (Error mounting /var/lib/nova/instances/7bf0f046-69a7-4c5f-990b-9cbfaf3b0b55/disk with l
ibguestfs (cannot find any suitable libguestfs supermin, fixed or old-style appliance on LIBGUESTFS_PATH (search path: /usr/l
ib/guestfs)))

參考http://libguestfs.org/guestfs-faq.1.html

libguestfs: error: cannot find any suitable libguestfs supermin, fixed or old-style appliance on LIBGUESTFS_PATH

febootstrap-supermin-helper: ext2: parent directory not found

supermin-helper: ext2: parent directory not found

[This issue is fixed permanently in libguestfs ≥ 1.26.]

If you see any of these errors on Debian/Ubuntu, you need to run the following command:

sudo update-guestfs-appliance

我運行了update-guestfs-appliance，但是仍是無論用，那就升級吧

ubuntu 14.04的repository裏面的guestfs是1.24，因而我下載了最新的1.26，而後從代碼安裝

wget http://libguestfs.org/download/1.26-stable/libguestfs-1.26.0.tar.gz

tar xvzf libguestfs-1.26.0.tar.gz

cd libguestfs-1.26.0

apt-get install build-essential

首先用apt-get purge將原來安裝的全部的guestfs的package都刪除

爲了可以編譯成功，安裝了下面的包

apt-get install libncurses5 libncurses5-dev ocaml ocaml-findlib-wizard ocaml-findlib libfindlib-ocaml libfindlib-ocaml-dev pk e2fslibs e2fslibs-dev gperf flex bison libpcre3 libpcre3-dev libaugeas-dev ncurses-base libcurses-ocaml ncurses-bin ncurses-dev erlang-dev

固然可能還不侷限於這些包，在configure, make, make install的過程當中，遇到沒有的包，就apt-cache search，而後install

在configure libguestfs以前，supermin也須要升級

wget http://libguestfs.org/download/supermin/supermin-5.1.8.tar.gz

tar xvzf supermin-5.1.8.tar.gz

cd supermin-5.1.8

./configure

make

make install

上面的有些包是爲安裝supermin準備的。

在configure libguestfs以前，要export LIBS="-lncurses"，要不他會找不到tgetent和tgetnum

並且configure的是，會對qemu的版本作檢查

在configure.ac中

772 #        AC_MSG_CHECKING([for $QEMU version >= 1])
773 #        if $QEMU -version | grep -sq 'version @<:@1-@:>@'; then
774 #            AC_MSG_RESULT([yes])
775 #        else
776 #            AC_MSG_RESULT([no])
777 #            AC_MSG_FAILURE([$QEMU version must be >= 1.0.])
778 #        fi

這裏要求qemu的版本必定是1.x

很不幸

# qemu-system-x86_64 --version
QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1), Copyright (c) 2003-2008 Fabrice Bellard

個人是2.x，反而報錯誤說個人版本老了，沒辦法，只好註釋掉

修改了configure.ac，因而須要安裝apt-get install autoconf

運行autoconf從新生成configure文件

下面configure libguestfs，須要指定qemu

./configure --with-qemu="qemu-system-x86_64"

中間有任何的錯誤都必須google，而後讓他過，不能忽略，由於libguestfs的工具和包太多，說不定哪一個不起做用，都會影響文件注入

make

make install

中間有.so, .a衝突，要把原來的弄掉

cd /

find . -name "dllmlguestfs.so"

cd /usr/lib/ocaml/stublibs/

mv dllmlguestfs.so dllgraphics.so.back

從新make install，直到正常成功結束。

安裝成功了，試一把

virt-ls -a ubuntutest.img /home/openstack

彷佛成功

guestfish -a ubuntutest.img

裏面運行run

彷佛也成功

就基本能夠了

接下來建立虛擬機，帶文件注入

可仍是報這個錯誤

2014-07-10 08:11:53.226 2550 TRACE nova.compute.utils [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1355, in _build_instance
2014-07-10 08:11:53.226 2550 TRACE nova.compute.utils [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]     filter_properties, bdms, legacy_bdm_in_spec)
2014-07-10 08:11:53.226 2550 TRACE nova.compute.utils [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1401, in _reschedule_or_error
2014-07-10 08:11:53.226 2550 TRACE nova.compute.utils [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]     self._log_original_error(exc_info, instance_uuid)
2014-07-10 08:11:53.226 2550 TRACE nova.compute.utils [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
2014-07-10 08:11:53.226 2550 TRACE nova.compute.utils [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]     six.reraise(self.type_, self.value, self.tb)
2014-07-10 08:11:53.226 2550 TRACE nova.compute.utils [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1396, in _reschedule_or_error
2014-07-10 08:11:53.226 2550 TRACE nova.compute.utils [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]     bdms, requested_networks)
2014-07-10 08:11:53.226 2550 TRACE nova.compute.utils [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2125, in _shutdown_instance
2014-07-10 08:11:53.226 2550 TRACE nova.compute.utils [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]     requested_networks)
2014-07-10 08:11:53.226 2550 TRACE nova.compute.utils [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
2014-07-10 08:11:53.226 2550 TRACE nova.compute.utils [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]     six.reraise(self.type_, self.value, self.tb)
2014-07-10 08:11:53.226 2550 TRACE nova.compute.utils [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2115, in _shutdown_instance
2014-07-10 08:11:53.152 2550 TRACE nova.compute.manager [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]     instance, network_info, admin_pass, files, suffix)
2014-07-10 08:11:53.152 2550 TRACE nova.compute.manager [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2539, in _inject_data
2014-07-10 08:11:53.152 2550 TRACE nova.compute.manager [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]     instance=instance)
2014-07-10 08:11:53.152 2550 TRACE nova.compute.manager [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
2014-07-10 08:11:53.152 2550 TRACE nova.compute.manager [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]     six.reraise(self.type_, self.value, self.tb)
2014-07-10 08:11:53.152 2550 TRACE nova.compute.manager [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]   File /usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2533, in _inject_data
2014-07-10 08:11:53.152 2550 TRACE nova.compute.manager [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]     mandatory=('files',))
2014-07-10 08:11:53.152 2550 TRACE nova.compute.manager [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]   File "/usr/lib/python2.7/dist-packages/nova/virt/disk/api.py", line 353, in inject_data
2014-07-10 08:11:53.152 2550 TRACE nova.compute.manager [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]     fs.setup()
2014-07-10 08:11:53.152 2550 TRACE nova.compute.manager [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]   File  /usr/lib/python2.7/dist-packages/nova/virt/disk/vfs/guestfs.py", line 132, in setup
2014-07-10 08:11:53.152 2550 TRACE nova.compute.manager [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672]     {'imgfile': self.imgfile, 'e': e})
2014-07-10 08:11:53.152 2550 TRACE nova.compute.manager [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672] NovaException: Error mounting /var/lib/nova/instances/e92f42dd-4164-4dff-a6a7-d42bd04be672/disk with libguestfs (/usr/local/bin/supermin exited w
ith error status 1.
2014-07-10 08:11:53.152 2550 TRACE nova.compute.manager [instance: e92f42dd-4164-4dff-a6a7-d42bd04be672] To see full error messages you may need to enable debugging.

爲何，不明覺厲

用pdb debug一把(在root下)

python -m pdb nova-pdb-compute.py --config-file=/etc/nova/nova.conf --config-file=/etc/nova/nova-compute.conf

# cat nova-pdb-compute.py
#! /usr/bin/python
# PBR Generated from u'console_scripts'

import sys
import pdb

from nova.cmd.compute import main

if __name__ == "__main__":
    sys.exit(main())

就是從/usr/bin/nova-compute拷貝過來，前面加上import pdb

單步運行，居然經過了，想不出什麼區別

忽然看到

# ps aux | grep nova-compute
root     22745  0.0  0.0  10464   928 pts/3    S+   19:37   0:00 grep --color=auto nova-compute
nova     29488  1.0  0.1 1761032 75100 ?       Ssl  17:25   1:20 /usr/bin/python /usr/bin/nova-compute --config-file=/etc/nova/nova.conf --config-file=/etc/nova/nova-compute.conf

service nova-compute start啓動的進程不是以root運行的，而是以nova用戶運行的

因而service nova-compute stop

而後在root下運行

/usr/bin/python /usr/bin/nova-compute --config-file=/etc/nova/nova.conf --config-file=/etc/nova/nova-compute.conf > nova-compute.log 2>&1 &

果真成功了。

可見普通用戶運行guestfish仍是有問題的，上面的全部操做都是在root下，因而切換到普通用戶

cliu8:~/images$ guestfish -a ubuntutest.img

Welcome to guestfish, the guest filesystem shell for
editing virtual machine filesystems and disk images.

Type: 'help' for help on commands
      'man' to read the manual
      'quit' to quit the shell

><fs> run
libguestfs: warning: current user is not a member of the KVM group (group ID 112). This user cannot access /dev/kvm, so libguestfs may run very slowly. It is recommended that you 'chmod 0666 /dev/kvm' or add the current user to the KVM group (you might need to log out and log in again).
libguestfs: error: /usr/local/bin/supermin exited with error status 1.
To see full error messages you may need to enable debugging.
See http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs

哈哈，在普通用戶下，果真run不起appliance來

usermod -G kvm -a nova
usermod -G kvm -a cliu8
usermod -G root -a cliu8  
usermod -G root -a nova

將普通用戶和nova都添加到kvm, root的group

打開debug，再運行guestfish

cliu8:~/images$ export LIBGUESTFS_DEBUG=1
cliu8:~/images$ export LIBGUESTFS_TRACE=1
cliu8:~/images$ guestfish -a ubuntutest.img
libguestfs: trace: set_verbose true
libguestfs: trace: set_verbose = 0
libguestfs: create: flags = 0, handle = 0x196ba50, program = guestfish
libguestfs: trace: set_pgroup true
libguestfs: trace: set_pgroup = 0
libguestfs: trace: add_drive "ubuntutest.img"
libguestfs: trace: add_drive = 0

Welcome to guestfish, the guest filesystem shell for
editing virtual machine filesystems and disk images.

Type: 'help' for help on commands
      'man' to read the manual
      'quit' to quit the shell

><fs> run
libguestfs: trace: launch
libguestfs: trace: get_tmpdir
libguestfs: trace: get_tmpdir = "/tmp"
libguestfs: trace: version
libguestfs: trace: version = <struct guestfs_version *>
libguestfs: trace: get_backend
libguestfs: trace: get_backend = "direct"
libguestfs: launch: program=guestfish
libguestfs: launch: version=1.26.0
libguestfs: launch: backend registered: unix
libguestfs: launch: backend registered: uml
libguestfs: launch: backend registered: direct
libguestfs: launch: backend=direct
libguestfs: launch: tmpdir=/tmp/libguestfsE2FajZ
libguestfs: launch: umask=0002
libguestfs: launch: euid=1004
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/var/tmp"
libguestfs: [00000ms] begin building supermin appliance
libguestfs: [00000ms] run supermin
libguestfs: command: run: /usr/local/bin/supermin
libguestfs: command: run: \ --build
libguestfs: command: run: \ --verbose
libguestfs: command: run: \ --if-newer
libguestfs: command: run: \ --lock /var/tmp/.guestfs-1004/lock
libguestfs: command: run: \ --copy-kernel
libguestfs: command: run: \ -f ext2
libguestfs: command: run: \ --host-cpu x86_64
libguestfs: command: run: \ /usr/local/lib/guestfs/supermin.d
libguestfs: command: run: \ -o /var/tmp/.guestfs-1004/appliance.d
supermin: version: 5.1.8
supermin: package handler: debian/dpkg
supermin: acquiring lock on /var/tmp/.guestfs-1004/lock
supermin: build: /usr/local/lib/guestfs/supermin.d
supermin: build: visiting /usr/local/lib/guestfs/supermin.d/base.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/local/lib/guestfs/supermin.d/daemon.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/local/lib/guestfs/supermin.d/excludefiles type uncompressed excludefiles
supermin: build: visiting /usr/local/lib/guestfs/supermin.d/hostfiles type uncompressed hostfiles
supermin: build: visiting /usr/local/lib/guestfs/supermin.d/init.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/local/lib/guestfs/supermin.d/packages type uncompressed packages
supermin: build: visiting /usr/local/lib/guestfs/supermin.d/udev-rules.tar.gz type gzip base image (tar)
supermin: build: 161 packages, including dependencies
supermin: build: 8030 files
supermin: build: 8030 files, after removing unreadable files
supermin: build: 5362 files, after matching excludefiles
supermin: build: 5369 files, after adding hostfiles
supermin: build: 5372 files, after munging
supermin: kernel: picked kernel vmlinuz-3.13.0-27-generic
cp: cannot open '/boot/vmlinuz-3.13.0-27-generic' for reading: Permission denied
supermin: cp -p '/boot/vmlinuz-3.13.0-27-generic' '/var/tmp/.guestfs-1004/appliance.d.383nplf0/kernel': command failed, see earlier errors
libguestfs: error: /usr/local/bin/supermin exited with error status 1, see debug messages above
libguestfs: trace: launch = -1 (error)
><fs> quit

居然仍是不行

最後終於發現

http://libguestfs.org/guestfs-faq.1.html

DOWNLOADING, INSTALLING, COMPILING LIBGUESTFS

Debian and Ubuntu After installing libguestfs you need to do:

sudo update-guestfs-appliance

On Ubuntu only:

sudo chmod 0644 /boot/vmlinuz*

You may need to add yourself to the kvm group:

sudo usermod -a -G kvm yourlogin

真是不看文檔，吃虧在眼前啊

趕忙

chmod 0644 /boot/vmlinuz*

而後運行guestfish -a ubuntutest.img，裏面run，哈哈成功了

因而能夠再nova用戶下運行nova-compute了

service nova-compute start

再order一個機器，帶文件注入的

成了

# ip netns exec qrouter-26a45e0e-a58a-443b-a972-d62c0c5a1323 ssh -i openstack.pem ubuntu@192.168.0.33
The authenticity of host '192.168.0.33 (192.168.0.33)' can't be established.
ECDSA key fingerprint is 20:f6:58:a9:c0:ab:54:7e:5f:3b:1f:3c:75:c6:36:26.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.0.33' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 12.04.4 LTS (GNU/Linux 3.2.0-64-virtual x86_64)

* Documentation:  https://help.ubuntu.com/

  System information as of Thu Jul 10 11:48:11 UTC 2014

  System load:  0.0               Processes:           59
  Usage of /:   4.0% of 19.68GB   Users logged in:     0
  Memory usage: 3%                IP address for eth0: 192.168.0.33
  Swap usage:   0%

  Graph this data and manage this system at:
https://landscape.canonical.com/

  Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud

0 packages can be updated.
0 updates are security updates.

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

ubuntu@testinject15:~$ lsbootfromvolume1.xml