Openstack nova/glance涉及不少instance、image、snapshot的概念和操做,一直搞不清楚。其中不少跟qemu有關係,底層都是經過libvirt調用kvm/qemu接口實現的。php
因此但願從qemu命令入手,對此有所瞭解。css
官網:
http://wiki.qemu.org/Main_Page
http://wiki.qemu.org/Manualhtml
https://en.wikibooks.org/wiki/QEMU/Linux
http://my.oschina.net/kelvinxupt/blog/265108node
[felix@centos65 ~]$ wget http://wiki.qemu-project.org/download/qemu-2.3.1.tar.bz2
[felix@centos65 ~]$ tar xjvf qemu-2.3.1.tar.bz2
[felix@centos65 ~]$ cd qemu-2.3.1
[felix@centos65 qemu-2.3.1]$ sudo yum install zlib-devel
[felix@centos65 qemu-2.3.1]$ sudo yum install glib2-devel -y
[felix@centos65 qemu-2.3.1]$ sudo yum install install autoconf automake libtool
[felix@centos65 qemu-2.3.1]$ ./configure --enable-kvm --enable-debug --enable-vnc --target-list="x86_64-softmmu"
[felix@centos65 qemu-2.3.1]$ sudo make install
.....
install -d -m 0755 "/usr/local/bin"
install -c -m 0755 qemu-system-x86_64 "/usr/local/bin"
[felix@centos65 qemu-2.3.1]$ qemu-
qemu-ga qemu-img qemu-io qemu-nbd qemu-system-x86_64
[felix@centos65 qemu-2.3.1]$ which qemu-img
/usr/local/bin/qemu-img
[felix@centos65 qemu-2.3.1]$ which qemu-system-x86_64
/usr/local/bin/qemu-system-x86_64
參考1python
Snapshots in QEMU are images that refer to an original image using Redirect-on-Write to avoid changing the original imagelinux
參考2
Copy-on-Write(寫時拷貝) ios
所以:git
優勢:github
缺點:json
Redirect-on-write(寫時重定向)
和copy-on-write類似,但沒有兩次寫的動做:
優勢:
缺點:
(原文還有其餘方式,這裏不作記錄)
思考:
nova boot instance用了qemu create with backing_file(?)即redirect-on-write方式,instance/_base/xxxxx(backing_file)就是original data,instance/yyyy/disk就至關於snapshot data。當nova存儲後端爲ceph時,沒有backing_file和disk文件,那是怎麼回事?
Update:
# libvirt.xml
<disk type="network" device="disk">
<driver type="raw" cache="writeback"/>
<source protocol="rbd" name="vms/eb511866-3b8f-40e0-9650-ea040b439a7f_disk">
<host name="10.254.3.111" port="6789"/>
</source>
<auth username="cinder">
<secret type="ceph" uuid="ea60f8ea-18d1-4dba-afa9-6528b1685100"/>
</auth>
<target bus="virtio" dev="vda"/>
</disk>
op@ubuntu-op:/home/openstack/workspace/data/nova/instances$ qemu-img info rbd:vms/eb511866-3b8f-40e0-9650-ea040b439a7f_disk:id=cinder:conf=/etc/ceph/ceph.conf
image: rbd:vms/eb511866-3b8f-40e0-9650-ea040b439a7f_disk:id=cinder:conf=/etc/ceph/ceph.conf
file format: raw
virtual size: 40G (42949672960 bytes)
disk size: unavailable
cluster_size: 4194304
對於raw格式的image啓動instance,也不是用的redirect-on-write方式吧(由於沒有backing_file)?
Update:用Openstack自帶的cirros建立的虛機沒有backing_file(由於它用kernel/ramdisk?)
[admin@maqi-openstack instances]$ ls d8e20388-2ff0-4667-a9b3-29921bf415be/
console.log disk.config disk.info kernel libvirt.xml ramdisk
可是自建的raw格式glance image,啓動的虛機仍是有backing_file的
nova image-create(Create a new image by taking a snapshot of a running server)是怎麼回事?
Update:這個就是nova snapshot功能,它和qemu-img snapshot不是一回事。
lihao:(nova/virt/libvirt/driver.py _live_snapshot)
建立鏡像的時候,可使用-b指定backing_file,也叫base image。這樣作的好處:
https://en.wikibooks.org/wiki/QEMU/Images
raw:
(default) the raw format is a plain binary image of the disc image, and is very portable. On filesystems that support sparse files, images in this format only use the space actually used by the data recorded in them.
qcow2:
QEMU Copy-On-Write format with a range of special features, including the ability to take multiple snapshots, smaller images on filesystems that don’t support sparse files, optional AES encryption, and optional zlib compression
https://people.gnome.org/~markmc/qcow-image-format.html
http://www.ibm.com/developerworks/cn/linux/1409_qiaoly_qemuimgages/
QEMU Guest Agent
之後再看。
http://manned.org/qemu-io/8ccaf7b2
qemu-io is a command line utility to exercise the QEMU I/O path.
之後再看。
[admin@maqi-centos7 ~]$ qemu-nbd -h
Usage: qemu-nbd [OPTIONS] FILE
QEMU Disk Network Block Device Server
NBD:Network Block Device
簡單來講,qemu-nbd能夠把一個qemu disk image經過nbd方式掛載到系統上:
[admin@maqi-centos7 ~]$ sudo qemu-nbd --connect=/dev/nbd0 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2
Failed to open /dev/nbd0: No such file or directory[admin@maqi-centos7 ~]$ sudo touch /dev/nbd0
[admin@maqi-centos7 ~]$ sudo qemu-nbd --connect=/dev/nbd0 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2
nbd.c:nbd_init():L571: Failed to set NBD socket
須要kernel支持nbd:http://www.aixchina.net/Question/123287?p=1
找另外一個環境試驗:
felix@ubuntu14-home:~|⇒ sudo qemu-nbd -c /dev/nbd0 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2
felix@ubuntu14-home:~|⇒ ll /dev/nbd0
brw-rw---- 1 root disk 43, 0 9月 21 03:05 /dev/nbd0
felix@ubuntu14-home:~|⇒ sudo fdisk /dev/nbd0
Command (m for help): p
Disk /dev/nbd0: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders, total 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00010f6d
Device Boot Start End Blocks Id System
/dev/nbd0p1 * 2048 20971519 10484736 83 Linux
Command (m for help): q
felix@ubuntu14-home:~|⇒ sudo mount /dev/nbd0p1 /mnt
felix@ubuntu14-home:~|⇒ ll /mnt/home
total 4.0K
drwx------ 4 500 500 4.0K 2月 10 2015 admin
felix@ubuntu14-home:~|⇒ mount | grep nbd
/dev/nbd0p1 on /mnt type ext4 (rw)
felix@ubuntu14-home:~|⇒ sudo umount /mnt
felix@ubuntu14-home:~|⇒ sudo qemu-nbd -d /dev/nbd0
/dev/nbd0 disconnected
這種是本地的鏡像,固然也能夠經過IP地址掛載網絡上的遠程鏡像。
另外能夠參考:
https://www.kumari.net/index.php/system-adminstration/49-mounting-a-qemu-image
http://edoceo.com/cli/qemu
https://en.wikibooks.org/wiki/QEMU/Images#Mounting_an_image_on_the_host
http://smilejay.com/2012/11/how-to-mount-a-qcow2-image/ (qcow2格式和raw格式方法不一樣)
Openstack中有qemu-nbd相關的code:
https://github.com/openstack/nova/blob/master/nova/virt/disk/mount/nbd.py
qemu-img allows you to create, convert and modify images offline. It can handle all image formats supported by QEMU.
使用格式:qemu-img command [command options]
支持如下命令
check [-q] [-f fmt] [–output=ofmt] [-r [leaks | all]] [-T src_cache] filename
對鏡像作一致性檢查(consistency check),只支持」qcow2」, 「qed」 and 「vdi」
[admin@maqi-centos7 ~]$ qemu-img check bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 -- output=json
{
"image-end-offset": 616562688,
"compressed-clusters": 22160,
"total-clusters": 163840,
"check-errors": 0,
"allocated-clusters": 24398,
"filename": "bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2",
"format": "qcow2",
"fragmented-clusters": 22453
}
create [-q] [-f fmt] [-o options] filename [size]
建立鏡像。options表示打開additional feature,例如backing_file
:
If the option backing_file is specified, then the image will record only the differences from backing_file. No size needs to be specified in this case.
backing_file will never be modified unless you use the 「commit」 monitor command (or qemu-img commit).
nova中會用到這個參數:
# nova/virt/libvirt/utils.py
def create_cow_image(backing_file, path, size=None):
"""Create COW image Creates a COW image with the given backing file :param backing_file: Existing image on which to base the COW image :param path: Desired location of the COW image """
base_cmd = ['qemu-img', 'create', '-f', 'qcow2']
cow_opts = []
if backing_file:
cow_opts += ['backing_file=%s' % backing_file]
base_details = images.qemu_img_info(backing_file)
else:
base_details = None
if base_details and base_details.cluster_size is not None:
cow_opts += ['cluster_size=%s' % base_details.cluster_size]
if size is not None:
cow_opts += ['size=%s' % size]
if cow_opts:
# Format as a comma separated list
csv_opts = ",".join(cow_opts)
cow_opts = ['-o', csv_opts]
cmd = base_cmd + cow_opts + [path]
execute(*cmd)
這個方法最終會被nova/virt/libvirt/driver.py中的snapshot方法和nova/virt/libvirt/imagebackend.py中的Qcow2::create_image用到。
不使用backing_file:
[admin@maqi-centos7 qemu]$ qemu-img create -f qcow2 test.qcow 10G
[admin@maqi-centos7 qemu]$ qemu-img info test.qcow
image: test.qcow
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 196K
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
使用backing_file:
[admin@maqi-centos7 qemu]$ qemu-img create -f qcow2 -b bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 saved.qcow2
Formatting 'saved.qcow2', fmt=qcow2 size=10737418240 backing_file='bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2' encryption=off cluster_size=65536 lazy_refcounts=off
[admin@maqi-centos7 qemu]$ ls -ltrh
total 589M
-rw-r-----. 1 admin admin 589M Sep 27 03:27 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2
-rw-r--r--. 1 admin admin 193K Sep 27 06:42 saved.qcow2
[admin@maqi-centos7 qemu]$ qemu-img info saved.qcow2
image: saved.qcow2
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 196K
cluster_size: 65536
backing file: bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2
Format specific information:
compat: 1.1
lazy refcounts: false
commit [-q] [-f fmt] [-t cache] filename
Commit the changes recorded in filename in its base image or backing file.
If the backing file is smaller than the snapshot, then the backing file will be resized to be the same size as the snapshot.
compare [-f fmt] [-F fmt] [-T src_cache] [-p] [-q] [-s] filename1 filename2
Check if two images have the same content. You can compare images with different format or settings.
convert [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-o options] [-s snapshot_name] [-S sparse_size] filename [filename2 […]] output_filename
Convert the disk image filename or a snapshot snapshot_name to disk image output_filename using format output_fmt. It can be optionally compressed (「-c」 option) or use any format specific options like encryption (「-o」 option).
You can use the backing_file option to force the output image to be created as a copy on write image of the specified base image; the backing_file should have the same content as the input’s base image, however the path, image format, etc may differ.
If the 「-n」 option is specified, the target volume creation will be skipped. This is useful for formats such as 「rbd」 if the target volume has already been created with site specific options that cannot be supplied through qemu-img.
# qcow2 -> raw
[admin@maqi-centos7 ~]$ qemu-img convert -O raw bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.raw
# raw -> qcow2
[admin@maqi-centos7 ~]$ qemu-img convert -O qcow2 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.raw bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2.2
# raw -> qcow2 (compress)
[admin@maqi-centos7 ~]$ qemu-img convert -O qcow2 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.raw bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2.3 -c
[admin@maqi-centos7 ~]$ ls -ltrh bcec*
-rw-r-----. 1 admin admin 588M Sep 26 03:59 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2
-rw-r--r--. 1 admin admin 10G Sep 26 11:12 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.raw
-rw-r--r--. 1 admin admin 1.5G Sep 26 11:15 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2.2
-rw-r--r--. 1 admin admin 586M Sep 26 11:18 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2.3
[admin@maqi-centos7 ~]$ qemu-img info bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2
image: bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 588M
cluster_size: 65536
Format specific information:
compat: 0.10
[admin@maqi-centos7 ~]$ qemu-img info bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2.2
image: bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2.2
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 1.5G
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
[admin@maqi-centos7 ~]$ qemu-img info bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2.3
image: bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2.3
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 585M
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
[admin@maqi-centos7 ~]$ qemu-img info bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.raw
image: bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.raw
file format: raw
virtual size: 10G (10737418240 bytes)
disk size: 1.5G
1.virtual_size
都是同樣的,表示用該image裝的系統的disk大小
2.disk size
表示image實際的大小
info [-f fmt] [–output=ofmt] [–backing-chain] filename
查看鏡像信息
[root@node-7 instances]# pwd
/var/lib/nova/instances
[root@node-7 instances]# ll
total 8
drwxr-xr-x 2 nova nova 69 Sep 25 04:22 60a7a481-baea-47b4-9e95-f28c2fb2e2e1
drwxr-xr-x 2 nova nova 69 Sep 22 03:18 6839e37f-d3b6-49f5-bc50-8393daf5f918
drwxr-xr-x 2 nova nova 53 Sep 26 08:43 _base
-rw-r--r-- 1 nova nova 40 Sep 26 11:23 compute_nodes
drwxr-xr-x 2 nova nova 69 Sep 25 03:08 e8c36933-fd3f-415d-8b50-bb365afb9a12_resize
drwxr-xr-x 2 nova nova 69 Sep 25 08:36 f7345e0d-0f02-4ad8-9b3d-9082a433ecb6
drwxr-xr-x 2 nova nova 4096 Sep 25 02:16 locks
drwxr-xr-x 2 nova nova 6 Aug 4 02:19 snapshots
[root@node-7 instances]# qemu-img info f7345e0d-0f02-4ad8-9b3d-9082a433ecb6/disk
image: f7345e0d-0f02-4ad8-9b3d-9082a433ecb6/disk
file format: qcow2
virtual size: 40G (42949672960 bytes)
disk size: 8.4M
cluster_size: 65536
backing file: /var/lib/nova/instances/_base/da9c151216e24ec6136b139737006287fa476882
Format specific information:
compat: 1.1
lazy refcounts: false
[root@node-7 instances]#
[root@node-7 instances]# qemu-img info /var/lib/nova/instances/_base/ da9c151216e24ec6136b139737006287fa476882
image: /var/lib/nova/instances/_base/da9c151216e24ec6136b139737006287fa476882
file format: raw
virtual size: 10G (10737418240 bytes)
disk size: 1.5G
[root@node-7 instances]# qemu-img info 60a7a481-baea-47b4-9e95-f28c2fb2e2e1/disk
image: 60a7a481-baea-47b4-9e95-f28c2fb2e2e1/disk
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 83M
cluster_size: 65536
backing file: /var/lib/nova/instances/_base/da9c151216e24ec6136b139737006287fa476882
Format specific information:
compat: 1.1
lazy refcounts: false
是否是這樣:
1)在Openstack中,用qcow2鏡像啓動的VM,都會有backing_file,而且這個backing_file是能夠共享的。(nova用ceph作後端時,看不到backing_file。見「其餘」)
2)nova會把這個qcow2鏡像convert成raw格式,來做爲backing_file(Yes)
map [-f fmt] [–output=ofmt] filename
Dump the metadata of image filename and its backing file chain.
snapshot [-q] [-l | -a snapshot | -c snapshot | -d snapshot] filename
List, apply, create or delete snapshots in image filename
[admin@maqi-centos7 ~]$ qemu-img snapshot -l bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2
[admin@maqi-centos7 ~]$ qemu-img snapshot -c snapshot-1 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress. qcow2
[admin@maqi-centos7 ~]$ qemu-img snapshot -l bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2
Snapshot list:
ID TAG VM SIZE DATE VM CLOCK
1 snapshot-1 0 2015-09-27 03:27:26 00:00:00.000
[admin@maqi-centos7 ~]$ qemu-img snapshot -c snapshot-2 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress. qcow2
[admin@maqi-centos7 ~]$ qemu-img snapshot -l bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2
Snapshot list:
ID TAG VM SIZE DATE VM CLOCK
1 snapshot-1 0 2015-09-27 03:27:26 00:00:00.000
2 snapshot-2 0 2015-09-27 03:27:46 00:00:00.000
[admin@maqi-centos7 ~]$ ll
total 4318928
-rw-r-----. 1 admin admin 616759808 Sep 27 03:27 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2
-rw-r--r--. 1 admin admin 1599733760 Sep 26 11:15 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress. qcow2.2
-rw-r--r--. 1 admin admin 613959168 Sep 26 11:18 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress. qcow2.3
-rw-r--r--. 1 admin admin 10737418240 Sep 26 11:12 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.raw
drwxrwxr-x. 15 admin admin 4096 Sep 8 00:25 devstack
-rw-rw-r--. 1 admin admin 2268 Aug 18 13:50 localrc
-rw-rw-r--. 1 admin admin 825 Sep 13 08:06 send.py
[admin@maqi-centos7 ~]$ qemu-img info bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2
image: bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 588M
cluster_size: 65536
Snapshot list:
ID TAG VM SIZE DATE VM CLOCK
1 snapshot-1 0 2015-09-27 03:27:26 00:00:00.000
2 snapshot-2 0 2015-09-27 03:27:46 00:00:00.000
Format specific information:
compat: 0.10
Q:snapshot-1,snapshot-2存儲在哪兒??
rebase [-q] [-f fmt] [-t cache] [-T src_cache] [-p] [-u] -b backing_file [-F backing_fmt] filename
改變image的backing_file
Changes the backing file of an image. Only the formats 「qcow2」 and 「qed」 support changing the backing file.
The backing file is changed to backing_file and (if the image format of filename supports this) the backing file format is changed to backing_fmt. If backing_file is specified as 「」 (the empty string), then the image is rebased onto no backing file (i.e. it will exist independently of any backing file).
[admin@maqi-centos7 qemu]$ qemu-img info saved.qcow2
image: saved.qcow2
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 196K
cluster_size: 65536
backing file: bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2
Format specific information:
compat: 1.1
lazy refcounts: false
[admin@maqi-centos7 qemu]$ qemu-img rebase -b bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 saved .qcow2
[admin@maqi-centos7 qemu]$ ls -ltrh
total 589M
-rw-r-----. 1 admin admin 589M Sep 27 03:27 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2
-rw-r--r--. 1 admin admin 193K Sep 27 06:43 saved.qcow2
[admin@maqi-centos7 qemu]$ qemu-img info saved.qcow2
image: saved.qcow2
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 196K
cluster_size: 65536
backing file: bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2
Format specific information:
compat: 1.1
lazy refcounts: false
[admin@maqi-centos7 qemu]$ qemu-img rebase -b "" saved.qcow2
[admin@maqi-centos7 qemu]$ ls -ltrh
total 2.6G
-rw-r-----. 1 admin admin 589M Sep 27 03:27 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2
-rw-r--r--. 1 admin admin 1.5G Sep 27 06:47 saved.qcow2
[admin@maqi-centos7 qemu]$ qemu-img info saved.qcow2
image: saved.qcow2
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 2.0G
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
resize [-q] filename [+ | -]size
Change the disk image as if it had been created with size.
Before using this command to shrink a disk image, you MUST use file system and partitioning tools inside the VM to reduce allocated file systems and partition sizes accordingly. Failure to do so will result in data loss!
After using this command to grow a disk image, you must use file system and partitioning tools inside the VM to actually begin using the new space on the device.
amend [-q] [-f fmt] [-t cache] -o options filename
Amends the image format specific options for the image file filename. Not all file formats support this operation.
支持格式:
vvfat vpc vmdk vhdx vdi sheepdog rbd raw host_cdrom host_floppy host_device file qed qcow2 qcow parallels nbd iscsi gluster dmg cloop bochs blkverify blkdebug
這兩個命令都用來啓動虛擬機,參數比較多,能夠google。
如下是nova起的一個虛擬機:
qemu 21604 1 0 Sep25 ? 00:05:04 /usr/bin/qemu-system-x86_64 -name instance-00000005 -S -machine pc-i440fx-2.0,accel=tcg,usb=off -m 512 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid d8e20388-2ff0-4667-a9b3-29921bf415be -smbios type=1,manufacturer=OpenStack Foundation,product=OpenStack Nova,version=12.0.0,serial=dae72fe0-cc06-4eb0-b779-7f25bfaf69df,uuid=d8e20388-2ff0-4667-a9b3-29921bf415be,family=Virtual Machine -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-00000005.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -kernel /home/openstack/workspace/data/nova/instances/d8e20388-2ff0-4667-a9b3-29921bf415be/kernel -initrd /home/openstack/workspace/data/nova/instances/d8e20388-2ff0-4667-a9b3-29921bf415be/ramdisk -append root=/dev/vdb console=tty0 console=ttyS0 no_timer_check -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/home/openstack/workspace/data/nova/instances/d8e20388-2ff0-4667-a9b3-29921bf415be/disk.config,if=none,id=drive-ide0-1-1,readonly=on,format=raw,cache=none -device ide-cd,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1 -drive file=/dev/disk/by-path/ip-10.133.6.83:3260-iscsi-iqn.2010-10.org.openstack:volume-ede0b75e-75c8-4c56-a372-052ebcb5e9db-lun-1,if=none,id=drive-virtio-disk1,format=raw,serial=ede0b75e-75c8-4c56-a372-052ebcb5e9db,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk1,id=virtio-disk1,bootindex=1 -netdev tap,fd=23,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:a0:1e:72,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/home/openstack/workspace/data/nova/instances/d8e20388-2ff0-4667-a9b3-29921bf415be/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -vnc 127.0.0.1:0 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on
nova都是調用的libvirt接口,跟上面命令執行的方式會有不一樣,還須要看看libvirt的接口。
Openstack中,nova/glance的後端都使用ceph時
1)glance的image目錄爲空:
op@ubuntu-op:/home/openstack/workspace/data$ ll glance/images/
total 8
drwxr-xr-x 2 op op 4096 9月 16 00:22 ./
drwxr-xr-x 4 op op 4096 9月 16 00:22 ../
op@ubuntu-op:/home/openstack/workspace/data$ ll glance/cache/
total 635236
drwxr-xr-x 5 op op 4096 9月 24 19:18 ./
drwxr-xr-x 4 op op 4096 9月 16 00:22 ../
-rw-r----- 1 op op 4979632 9月 16 18:03 373b5d20-c82a-4f90-b8e4-66bc21e37c4b
-rw-r----- 1 op op 25165824 9月 16 00:29 4d076e72-110d-4446-a20d-3716ad5e1d4a
-rw-r----- 1 op op 3740163 9月 16 18:03 62540238-1fee-46da-a288-17ab31b1985e
-rw-r----- 1 op op 616562688 9月 24 19:12 989d150b-ddb7-4079-83e4-2bbc2f01cd2c
-rw-r----- 1 op op 3072 9月 24 19:18 cache.db
drwxr-x--- 2 op op 4096 9月 24 19:12 incomplete/
drwxr-x--- 2 op op 4096 9月 16 00:23 invalid/
drwxr-x--- 2 op op 4096 9月 16 00:23 queue/
2)nova下也有_base目錄
op@ubuntu-op:/home/openstack/workspace/data/nova/instances$ ls
4d896e1c-30dc-4f76-97cc-b181953e622c ad0f2d70-4773-40b8-a628-1ca2f88b8531 eb511866-3b8f-40e0-9650- ea040b439a7f snapshots
5b14e497-09dc-4364-9862-adb5a2dc9241 _base f9b84ae7-0075-4e75-a873- e83e66d60683
7c303787-d2d8-43b1-a049-2cfc79507b2b compute_nodes locks
op@ubuntu-op:/home/openstack/workspace/data/nova/instances$ ll _base
total 1563504
drwxr-xr-x 2 op libvirtd 4096 9月 25 22:54 ./
drwxr-xr-x 11 op root 4096 9月 28 18:17 ../
-rw-r--r-- 1 op libvirtd 4979632 9月 16 18:03 373b5d20-c82a-4f90-b8e4-66bc21e37c4b
-rw-r--r-- 1 op libvirtd 3740163 9月 16 18:03 62540238-1fee-46da-a288-17ab31b1985e
-rw-r--r-- 1 op libvirtd 10737418240 9月 28 19:05 8eaa8bae3127538b02375cf3b2d000951de1b64e
op@ubuntu-op:/home/openstack/workspace/data$ ll nova/instances/5b14e497-09dc-4364-9862-adb5a2dc9241/
total 436
drwxr-xr-x 2 op libvirtd 4096 9月 27 04:37 ./
drwxr-xr-x 10 op root 4096 9月 25 17:54 ../
-rw-rw---- 1 libvirt-qemu kvm 6367 9月 25 17:56 console.log
-rw-r--r-- 1 libvirt-qemu kvm 419840 9月 25 17:54 disk.config
-rw-r--r-- 1 op libvirtd 106 9月 25 17:54 disk.info
-rw-r--r-- 1 op libvirtd 3008 9月 25 17:54 libvirt.xml
op@ubuntu-op:/home/openstack/workspace/data$ qemu-img info nova/instances/5b14e497-09dc-4364-9862- adb5a2dc9241/disk.config
image: nova/instances/5b14e497-09dc-4364-9862-adb5a2dc9241/disk.config
file format: raw
virtual size: 410K (419840 bytes)
disk size: 412K
op@ubuntu-op:/home/openstack/workspace/data$ qemu-img info nova/instances/5b14e497-09dc-4364-9862- adb5a2dc9241/disk.info
image: nova/instances/5b14e497-09dc-4364-9862-adb5a2dc9241/disk.info
file format: raw
virtual size: 512 (512 bytes)
disk size: 4.0K
libvirt.xml中關於disk的部分:
<disk type="network" device="disk">
<driver type="raw" cache="writeback"/>
<source protocol="rbd" name="vms/5b14e497-09dc-4364-9862-adb5a2dc9241_disk">
<host name="10.254.3.111" port="6789"/>
</source>
<auth username="cinder">
<secret type="ceph" uuid="ea60f8ea-18d1-4dba-afa9-6528b1685100"/>
</auth>
<target bus="virtio" dev="vda"/>
</disk>
<disk type="file" device="cdrom">
<driver name="qemu" type="raw" cache="none"/>
<source file="/home/openstack/workspace/data/nova/instances/5b14e497-09dc-4364-9862-adb5a2dc9241/disk. config"/>
<target bus="ide" dev="hdd"/>
</disk>
ceph中的image是raw格式,沒有backing_file:
op@ubuntu-op:/home/openstack/workspace/data/nova/instances$ qemu-img info rbd:vms/5b14e497-09dc-4364-9862- adb5a2dc9241_disk:id=cinder:conf=/etc/ceph/ceph.conf
image: rbd:vms/5b14e497-09dc-4364-9862-adb5a2dc9241_disk:id=cinder:conf=/etc/ceph/ceph.conf
file format: raw
virtual size: 40G (42949672960 bytes)
disk size: unavailable
cluster_size: 4194304
Internal Snapshots – A single qcow2 image file holds both the saved state
& the delta since that saved point. This can be further classified as :-
Internal disk snapshot: The state of the virtual disk at a given point in time. Both the snapshot & delta since the snapshot are stored in the same qcow2 file. Can be taken when the guest is ‘live’ or ‘offline’.
Internal system checkpoint: RAM state, device state & the disk-state of a running guest, are all stored in the same originial qcow2 file. Can be taken when the guest is running ‘live’.
External Snapshots – Here, when a snapshot is taken, the saved state will be stored in one file(from that point, it becomes a read-only backing file) & a new file(overlay) will track the deltas from that saved state. This can be further classified as :-
External disk snapshot: The snapshot of the disk is saved in one file, and the delta since the snapshot is tracked in a new qcow2 file. Can be taken when the guest is ‘live’ or ‘offline’.
External system checkpoint: Here, the guest’s disk-state will be saved in one file, its RAM & device-state will be saved in another new file (This implementation is in progress upstream libvirt, as of writing this).
VM State: Saves the RAM & device state of a running guest(not ‘disk-state’) to a file, so that it can be restored later. This simliar to doing hibernate of the system. (NOTE: The disk-state should be unmodified at the time of restoration.)