Usage of live migration feature (nova supports online migration and other links)

Environments

 

• OS: Ubuntu lucid 10.04 for both instances and host.
• NFS: nova-install-dir/instances has to be mounted by shared storage. (this version is tested using NFS)
• Instances: Instance must keep running without any EBS volume.

 

Pre-requisite settings

 

(A) shared
Storage As mentioned above, shared storage is inevitable for
Live_migration functionality. An example is NFS (my test environment ),
And example setting is described below. Prepare NFS server machine (
Nova-api server is OK), and add below line/etc/exports:

 nova-install-dir/instances a.b.c.d/255.255.0.0(rw,sync,fsid=0,no_root_squash

 

Where
"Nova-install-dir" is the directory which openstack is installed, and
Add appropriate ip address and netmask for "a. B. c. d/255.255.0.0", which
Shocould include compute nodes mounting this directory. Then restart nfs
Server.

 /etc/init.d/nfs-kernel-server restart
 /etc/init.d/idmapd restart

 

Also, at any compute nodes, add below line to/etc/fstab:

 a.b.c.d:/ DIR nfs4 defaults 0 0

 

Where
A. B. c. d is nfs server's ipaddress and "DIR" must be same
'Instances' (seenova. compute. manager for the default value)

Then try to mount at compute node. Check exported directory is successfully mounted.

 mount -a -v

 

If fail, try this at any hosts.

iptables -F

Also, check file/daemon permissions. We recommend CT any nova daemons are running as root.

root@openstack2-api:/opt/nova-2010.4# ps -ef | grep nova
root 5948 5904 9 11:29 pts/4 00:00:00 python /opt/nova-2010.4//bin/nova-api
root 5952 5908 6 11:29 pts/5 00:00:00 python /opt/nova-2010.4//bin/nova-objectstore
... (snip)

"Instances/" directory can be seen from server side:

root@openstack:~# ls -ld nova-install-dir/instances/
drwxr-xr-x 2 root root 4096 2010-12-07 14:34 nova-install-dir/instances/

Also, client (compute node) side:

# ls -ld nova-install-dir/instances/
drwxr-xr-x 2 root root 4096 2010-12-07 14:34 nova-install-dir/instances/

(B)
Libvirt settings In default configuration, this feature use simple tcp
Protocol (qemu + tcp: //). To use this protocol, below configuration is
Necessary.

Modify/etc/libvirt. conf:

before : #listen_tls = 0
after : listen_tls = 0

before : #listen_tcp = 1
after : listen_tcp = 1

Adding following line to/etc/libvirt. conf:

auth_tcp = "none"

Modify/etc/init/libvirt-bin.conf

 before : exec /usr/sbin/libvirtd -d
 after : exec /usr/sbin/libvirtd -d -l

Modify modify/etc/default/libvirt-bin

before :libvirtd_opts=" -d"
after :libvirtd_opts=" -d -l"

Then, restart libvirt

stop libvirt-bin && start libvirt-bin ps -ef | grep libvirt

Make sure you get the below result.

root@openstack2:/opt/nova-2010.2# ps -ef | grep libvirt
root 1145 1 0 Nov27 ? 00:00:03 /usr/sbin/libvirtd -d -l

If
You woshould like to use qemu + ssh or other protocol, change
"Live_migration_uri" flag. by adding "-- live_migration_uri"
/Etc/nova/OVA. conf and make appropriate settings described at http://libvirt.org /.

 

Usage

• To get a list of physical hosts,
  • Nova-manage host list
• To get a available pysical resource of each host,
  • Nova-manage host shows hostname
  • An
    Example result is below. The 1st line shows total amount of resource
    That the specified host has. The 2nd and latter lines show usage
    Resource per project. This command is created because admins can decide
    Which host shoshould be a destination of live migration.

  • HOST PROJECT cpu mem(mb) disk(gb)
    openstack2-c2 16 32232 878 > openstack2-c2 admin 1 2048 20

• For live migration,
  • Nova-manage instances live_migration ec2-id (I-xxxx) destination-host-name.
  • Once
    This command is executed, admins will check the status through
    Euca-describe-instances. The status is changed from 'running'
    'Mixing', and changed to 'running' when live migration finishes. Note
    That it depends on an environment how long it takes to live migration
    Finishes. If it finishes too fast, admins cannot see 'migrating 'status.
    

  • # euca-describe-instances
    Reservation:r-qlg3favp
    RESERVATION r-qlg3favp adminINSTANCE i-2ah453 ami-tiny 12.1.0.14 10.0.0.3
    migrating testkey (admin, openstack2-c2) 0 m1.small 2010-11-28 16:09:16

  • When
    Live migration finishes successfully, admin can check the part
    Euca-describe-instances which shows physical node information. (only
    When euca-describe-instances is executed by admin user) Admins also can
    Check live migration source compute node logfile which may show a log.

  • Live migration i-xxxx to DESTHOST finishes successfully.

 

Trouble shooting

When live migration fails somehow, error messages are shown:

• Scheduler logfile
• Source compute node logfile
• Dest compute node logfile

Online migration: http://wiki.openstack.org/UsageOfLiveMigration

CentOS installation: http://wiki.openstack.org/NovaInstall/CentOSNotes

NovaCategory: http://wiki.openstack.org/CategoryNova

Comparison between Nova and Eucalyptus: http://wiki.openstack.org/Nova/EucalyptusFeatureComparison? Highlight = % 28 \ bCategoryNova \ B % 29

Nova Overview: http://wiki.openstack.org/Nova? Highlight = % 28 \ bCategoryNova \ B % 29