Xen and SuSE 9.3

I have been playing with Xen a bit recently for quick prototyping of clusters of machines. It seems to work quite well on SuSE 9.3. The following is how I made it work

First of all you need to install the xen specific patches from your SuSE source media. They are all selectable through yast. For reference the ones I installed are shown below, the version numbers might not match the ones on the DVD because I have applied some recommended updates to my machine (including a newer kernel).

xen-doc-html-2.0.5c-4
xen-2.0.5c-4
kernel-xen-2.6.11.4-21.7

The next thing to do is to make yast update your boot loader config based on the new xen kernels. This can be done by running yast and choosing System -> Boot Loader Configuration -> Reset -> Propose New Configuration. Once it has written the new boot loader configuration you will need to select the XEN kernel as the default (Xen needs kernel support on the host as well as the virtual machines). When that has been done reboot your box with the new Xen equiped kernel.

Now you are running on the new Xen kernel, add xend to the system startup (chkconfig --add xend)

The next stage is to make yast install itself into a directory ready for use as the root filesystem of the Xen virtual machine. if you go into yast and choose Software -> Installation into Directory for XEN It will prompt you for some options, I accepted all the defaults apart from changing the Software to be a ‘minimum system’. You can see what my options looked like below:

Note: there seems to be an image to create an image but it didnt seem to work for me.

When this has completed you should have what looks like a root filesystem in /var/tmp/dirinstall now you need to get that into a ext2 filesystem image. This can be done as shown below:

# cd /u01/xen
# dd if=/dev/zero of=xen1-rootfs bs=1k seek=2048k count=1
# dd if=/dev/zero of=xen1-swap bs=1024k count=512
# mkswap xen1-swap
# mke2fs xen1-rootfs
# mount -t ext2 -o loop /u01/xen/xen1-rootfs /mnt/tmp
# cd /var/tmp/dirinstall
# find . | cpio -p /mnt/tmp
# umount /mnt/tmp

The commands above (in order) are doing the following:

  1. cd into the directory where I keep my xen disk images
  2. create a 2Gb sparse file image for the root filesystem
  3. create a 512 meg swap file image
  4. makes the swap image actually readable as swap to linux
  5. put a ext2 filesystem on the root image
  6. mount the root image on /mnt/tmp (this can be any temporary mount point of your choice)
  7. cd into the recently installed SuSE directory
  8. use find and cpio to copy the entire tree into the mounted image
  9. unmounting the image

Now that you have a image file you can create a Xen config file. mine is shown below:

# cat /etc/xen/xen1
kernel = "/boot/vmlinuz-2.6.11.4-21.7-xen"
memory = 64
name = "xen1"
nics = 1
vif = [ 'bridge=xen-br0' ]
disk = ['file:/u01/xen/xen1-root,sda1,w','file:/u01/xen/xen1-swap,sda2,w']
root = "/dev/sda1 rw"
hostname= "xen1"

Note the kernel line must match whichever version you are running in /boot and the paths to the disk images must match your locations. This file (on my system) was called xen1 and was in /etc/xen/

Now you are almost ready to start your virtual machine, before I started mine I added a fstab, copied over my passwd, group and shadow files and created a config file for eth0. To anything to your disk image you must first mount it:

# mount -t ext2 -o loop /u01/xen/xen1-rootfs /mnt/tmp

and copy across a few useful files:

# cp /etc/{shadow,group,passwd,resolv.conf} /mnt/tmp/etc

I also created a /etc/fstab file inside my disk image which looked like this:

/dev/sda1       /       ext3    errors=remount-ro       0       1
/dev/sda2       none    swap    sw                      0       0
proc            /proc   proc    defaults                0       0

Also create yourself a ifcfg-eth0 file so that networking is brought up on startup, mine looks like the one below change the ip address etc to suit:

# cat /mnt/tmp/etc/sysconfig/network/ifcfg-eth0
BOOTPROTO='static'
BROADCAST='192.168.254.255'
IPADDR='192.168.254.52'
MTU=''
NAME='Ethernet Network Card'
NETMASK='255.255.255.0'
NETWORK='192.168.254.0'
REMOTE_IPADDR=''
STARTMODE='auto'
USERCONTROL='no'
_nm_name='static-0'

When that is done, umount the disk image (remember to cd out of any directory below /mnt/tmp first):

# umount /mnt/tmp

Now you are ready to start your Xen virtual domain:

# xm create xen1 -c

If all goes well you should see what looks like a kernel booting and it will drop you at a login prompt, because we copied over your local passwd and shadow files you should be able to just login. There are some services which need turning off because they dont play well inside the VM (like the HW clock etc) but you can simply disable these by running yast when your virtual machine has booted.

There is a whole lot of extra stuff you can do with xen but all of that is outside scope of this document, I wanted to keep this as a short ‘How I made it work on Suse 9.3’ for more information check the documentation linked from the Xen home page at http://www.cl.cam.ac.uk/Research/SRG/netos/xen/

Mirroring SuSE

I find myself having to mirror bits of the SuSE install tree and updates quite often. The problem is I typically do not need all the .iso files, the src rpms and the debuginfo packages, so I use the command below to exclude them from my rsync:

*.iso
*-debuginfo-*
*/src/*
*-ar-*
*-ca-*
*-cs-*
*-da-*
*-de-*
*-el-*
*-es-*
*-et-*
*-fi-*
*-fr-*
*-hu-*
*-it-*
*-ja-*
*-ko-*
*-nl-*
*-pl-*
*-pt-*
*-ru-*
*-sk-*
*-sl-*
*-sv-*
*-tr-*
*-zh-*
*-af-*
*-bg-*
*-br-*
*-cy-*
*-en-*
*-eo-*
*-fo-*
*-ga-*
*-hr-*
*-is-*
*-nn-*
*-ro-*
*-nb-*

hint:You can paste the above list into a text file and pass it to rsync with the --exclude-from option

Also note that from the UK, I’ve found ftp.sh.cvut.cz to be a consistently fast mirror

Hacking the SUSE boot.iso

I’ve recently been using autoyast quite a bit to automate builds of new SUSE this is nice and easy if the machine network boots, you just use pxelinux but if the machine does not support PXE booting you have to boot from the mini boot.iso included with SUSE and then manually type in the install= and autoyast= kernel parameters.

To save time and the hassle of doing this I have built a custom SUSE 9.2 mini boot.iso with a new grub option which includes all of the information necessary. Below is the steps involved:

  1. get the mini boot iso from a SUSE mirror, the filename I grabbed was /pub/suse/i386/9.2/iso/SUSE-Linux-9.2-mini-installation.iso
  2. mount the iso:
    # mount -t iso9660 -o loop SUSE-Linux-9.2-mini-installation.iso /mnt/tmp
    
  3. copy the contents to a temporary directory and set them to be writeable:
    # mkdir /tmp/boot-iso
    # cp -av /mnt/tmp/* /tmp/boot-iso/
    # chmod -R 700 /tmp/boot-iso/*
    
  4. edit the boot loader config in boot/loader/isolinux.cfg. I added the following as the second option:
    label cmlinux
      kernel linux
      append initrd=initrd ramdisk_size=65536 splash=silent showopts \
       textmode=1 install=http://192.168.22.250/install/SUSE/i386/9.2/ \
       autoyast=http://192.168.22.250/install/SUSE/autoinstall/repository/ \
       textmode=1
    

    note: the lines are broken here for clarity make sure the append line is all on one line and the slashes are removed

    note: the url’s specified in the install= and autoyast= parameters are specific to our configuration, you will need to edit as appropriate for your own auto install setup

    you also need to edit the messages file which is what is displayed on screen immediately after boot, I added the following directly after the ‘Boot from harddisk’ option:

      cmlinux   - Use the CM autoyast installer
    
  5. now you are ready to make your bootable iso. Whilst inside the /tmp/boot-iso/ directory I ran the following:
    #mkisofs -R -b boot/loader/isolinux.bin -no-emul-boot \
     -boot-load-size 4 -boot-info-table -o ../custom-boot.iso .
    
  6. write this custom-boot.iso to a cd and boot from it. When you see the menu choose ‘cmlinux’ which will automatically set the install source to be the url in the install= parameter and set the autoyast repository to the url in the autoyast= parameter. If there is an entry in the rules.xml for the machine being booted an auto install should run automatically, if not you will be left with the option of doing a manual install

SuSE sudo weirdness

On SuSE 9.2 we have noticed that sudo didnt seem to work out of the box. Have tracked this to SuSE changing a default from the way it is shipped in the sudo package, from the man sudoers(5) man page:

targetpw    If set, sudo will prompt for the password the user specified
            by the -u flag (defaults to root) instead of the password of
            the invoking user.  This flag is off by default.

If you comment out the following line sudo will ask for the password of the current user NOT the user they are trying to run the command as:

Defaults targetpw

SuSE 9.1 ifconfig weirdness

Have just found out that ifconfig on SuSE 9.2 doesnt seem to show virtual intefaces:

message2:~ # ifconfig
eth0      Link encap:Ethernet  HWaddr 00:80:3F:2B:7B:60
          inet addr:192.168.0.200  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::280:3fff:fe2b:7b60/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:40203054 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4386872 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2879540080 (2746.1 Mb)  TX bytes:321973799 (307.0 Mb)
          Base address:0x2000 Memory:dd200000-dd220000

eth1      Link encap:Ethernet  HWaddr 00:80:3F:2B:7B:61
          inet addr:XXX.XXX.XXx.XXX  Bcast:XXX.XXX.XXx.XXX  Mask:255.255.255.224
          inet6 addr: fe80::280:3fff:fe2b:7b61/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:66946630 errors:0 dropped:0 overruns:0 frame:0
          TX packets:43575297 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:4317777 (4.1 Mb)  TX bytes:4011082191 (3825.2 Mb)
          Base address:0x2040 Memory:dd220000-dd240000

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:1692 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1692 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:113193 (110.5 Kb)  TX bytes:113193 (110.5 Kb)

message2:~ #

However if you use the ip command from the iproute2 package it shows that in fact eth1 does have some aliases:

message2:~ # ip addr
1: lo:  mtu 16436 qdisc noqueue
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: sit0:  mtu 1480 qdisc noqueue
    link/sit 0.0.0.0 brd 0.0.0.0
3: eth0:  mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:80:3f:2b:7b:60 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.200/24 brd 192.168.0.255 scope global eth0
    inet6 fe80::280:3fff:fe2b:7b60/64 scope link
       valid_lft forever preferred_lft forever
4: eth1:  mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:80:3f:2b:7b:61 brd ff:ff:ff:ff:ff:ff
    inet XXX.XXX.XXx.XXX/YY brd XXX.XXX.XXx.XXX scope global eth1
    inet XXX.XXX.XXx.XXX/YY brd XXX.XXX.XXx.XXX scope global secondary eth1
    inet6 fe80::280:3fff:fe2b:7b61/64 scope link
       valid_lft forever preferred_lft forever
message2:~ #

How weird is that. Versions shown below:

message2:~ # cat /etc/SuSE-release
SuSE Linux 9.1 (i586)
VERSION = 9.1
message2:~ # ifconfig -V
net-tools 1.60
ifconfig 1.42 (2001-04-13)
message2:~ #

NB:IP addresses changed to protect the innocent…