• BASH assigning command output into an array

    Often find myself needing to do this but never remember how…

    # flirble=(`ls -ltr ssh* | tail -3 | tr -s ' ' | cut -d ' ' -f 9` )
    # echo ${flirble[*]}
    ssh_host_key ssh_host_dsa_key.pub ssh_host_dsa_key
    # echo ${flirble[0]}
    ssh_host_key
    # echo ${flirble[1]}
    ssh_host_dsa_key.pub
    # echo ${flirble[2]}
    ssh_host_dsa_key
    # echo ${flirble[3]}
    
    # ls -ltr ssh* | tail -3
    -rw-------  1 robin cm-users 539 Aug 11 12:40 ssh_host_key
    -rw-r--r--  1 robin cm-users 614 Aug 11 12:40 ssh_host_dsa_key.pub
    -rw-------  1 robin cm-users 668 Aug 11 12:40 ssh_host_dsa_key
    

    This and more useful array manipulation can be found in Chapter 26 of the Advanced Bash Scripting Guide.


  • Sed regular expressions

    The sed regular expressions are essentially the same as the grep regular expressions. They are summarized below.

    <td>
      matches the beginning of the line
    </td>
    
    <td>
      matches the end of the line
    </td>
    
    <td>
      Matches any single character
    </td>
    
    <td>
      match arbitrarily many occurences of (character)
    </td>
    
    <td>
      Match 0 or 1 instance of (character)
    </td>
    
    <td>
      Match any character enclosed in [] (in this instance, a b c d e or f)<br /> ranges of characters such as <code>[a-z]</code> are permitted. The behaviour<br /> of this deserves more description. See the page on <a HREF="grep.html">grep</a><br /> for more details about the syntax of lists.
    </td>
    
    <td>
      Match any character <em>NOT</em> enclosed in [] (in this instance, any character other than a b c d e or f)
    </td>
    
    <td>
      Match m-n repetitions of (character)
    </td>
    
    <td>
      Match m or more repetitions of (character)
    </td>
    
    <td>
      Match n or less (possibly 0) repetitions of (character)
    </td>
    
    <td>
      Match exactly n repetitions of (character)
    </td>
    
    <td>
      Group operator.
    </td>
    
    <td>
      Backreference - matches nth group
    </td>
    
    <td>
      Matches expression1 or expression 2. Works with GNU sed, but this feature might not work with other forms of sed.
    </td>
    


  • 2 useful html things

    First, a nice css cheat sheet pdf:http://www.ilovejackdaniels.com/css_cheat_sheet.pdf

    And secondly a nice colour picker dashboard widget http://www.colourmod.com/


  • New del.icio.us links

    Down there on bottom of the right hand side there is a list of links from my del.icio.us bookmarks. It is only the ones tagged as useful so other things I tag wont turn up, like photography for example.

    I’ll try and make a specific page on this site of all of the links tagged as useful at some point.


  • Procinfo

    Just found this useful little tool for summerising machine information read from /proc. I can’t find a home page for it but it can be downloaded from ftp://ftp.cistron.nl/pub/people/svm/. It seems to ship by default with at least SUSE 9.3 and Fedora (Broken link http://fedora.redhat.com/) Core 4.

    Example output below

    [root@eddie ~]# procinfo
    Linux 2.6.11-1.1369_FC4 (bhcompile@decompose.build.redhat.com) \
    (gcc 4.0.0 20050525 ) #1 Thu Jun 2 22:55:56 EDT 2005 1CPU [eddie]
    
    Memory:      Total        Used        Free      Shared     Buffers
    Mem:        515372      446196       69176           0      113048
    Swap:      1048568         912     1047656
    
    Bootup: Sun Jul 17 08:47:22 2005    Load average: 0.43 0.22 0.08 1/138 11716
    
    user  :       3:36:41.09   1.7%  page in :        0
    nice  :       1:20:51.42   0.6%  page out:        0
    system:       3:40:27.39   1.7%  swap in :        0
    idle  :   8d 17:08:50.87  96.0%  swap out:        0
    uptime:   9d  1:46:23.08         context : 98642479
    
    irq  0: 784129991 timer                 irq  8:         1 rtc
    irq  1:         9 i8042                 irq  9:         0 acpi
    irq  2:         0 cascade [4]           irq 10:   6170092 CMI8738-MC6, ehci_hc
    irq  3:         5                       irq 11:         0 uhci_hcd:usb2, uhci_
    irq  4:         5                       irq 12:      2114 i8042
    irq  5:         0 uhci_hcd:usb1, yenta  irq 14:   2132585 ide0
    irq  6:         5                       irq 15:   2955294 ide1
    
    [root@eddie ~]#
    


  • DHCP search path

    I fyou need to send a DNS search path to DHCP clients along with all the other details it seems you do it like this (in /etc/dhcpd.conf):

    option domain-name "internal.usefulthings.org.uk usefulthings.org.uk";
    option domain-name-servers 192.168.8.1, 192.168.8.2;
    

    Took me a while to find that one!


  • HTML to Media wiki table converter

    Just found this (Broken link http://www.uni-bonn.de/~manfear/html2wiki-tables.php) great utility for converting html tables into the pipe format that media wiki uses.


  • Xen and SuSE 9.3

    I have been playing with Xen a bit recently for quick prototyping of clusters of machines. It seems to work quite well on SuSE 9.3. The following is how I made it work

    First of all you need to install the xen specific patches from your SuSE source media. They are all selectable through yast. For reference the ones I installed are shown below, the version numbers might not match the ones on the DVD because I have applied some recommended updates to my machine (including a newer kernel).

    xen-doc-html-2.0.5c-4
    xen-2.0.5c-4
    kernel-xen-2.6.11.4-21.7
    

    The next thing to do is to make yast update your boot loader config based on the new xen kernels. This can be done by running yast and choosing System -> Boot Loader Configuration -> Reset -> Propose New Configuration. Once it has written the new boot loader configuration you will need to select the XEN kernel as the default (Xen needs kernel support on the host as well as the virtual machines). When that has been done reboot your box with the new Xen equiped kernel.

    Now you are running on the new Xen kernel, add xend to the system startup (chkconfig --add xend)

    The next stage is to make yast install itself into a directory ready for use as the root filesystem of the Xen virtual machine. if you go into yast and choose Software -> Installation into Directory for XEN It will prompt you for some options, I accepted all the defaults apart from changing the Software to be a ‘minimum system’. You can see what my options looked like below:

    Note: there seems to be an image to create an image but it didnt seem to work for me.

    When this has completed you should have what looks like a root filesystem in /var/tmp/dirinstall now you need to get that into a ext2 filesystem image. This can be done as shown below:

    # cd /u01/xen
    # dd if=/dev/zero of=xen1-rootfs bs=1k seek=2048k count=1
    # dd if=/dev/zero of=xen1-swap bs=1024k count=512
    # mkswap xen1-swap
    # mke2fs xen1-rootfs
    # mount -t ext2 -o loop /u01/xen/xen1-rootfs /mnt/tmp
    # cd /var/tmp/dirinstall
    # find . | cpio -p /mnt/tmp
    # umount /mnt/tmp
    

    The commands above (in order) are doing the following:

    1. cd into the directory where I keep my xen disk images
    2. create a 2Gb sparse file image for the root filesystem
    3. create a 512 meg swap file image
    4. makes the swap image actually readable as swap to linux
    5. put a ext2 filesystem on the root image
    6. mount the root image on /mnt/tmp (this can be any temporary mount point of your choice)
    7. cd into the recently installed SuSE directory
    8. use find and cpio to copy the entire tree into the mounted image
    9. unmounting the image

    Now that you have a image file you can create a Xen config file. mine is shown below:

    # cat /etc/xen/xen1
    kernel = "/boot/vmlinuz-2.6.11.4-21.7-xen"
    memory = 64
    name = "xen1"
    nics = 1
    vif = [ 'bridge=xen-br0' ]
    disk = ['file:/u01/xen/xen1-root,sda1,w','file:/u01/xen/xen1-swap,sda2,w']
    root = "/dev/sda1 rw"
    hostname= "xen1"
    

    Note the kernel line must match whichever version you are running in /boot and the paths to the disk images must match your locations. This file (on my system) was called xen1 and was in /etc/xen/

    Now you are almost ready to start your virtual machine, before I started mine I added a fstab, copied over my passwd, group and shadow files and created a config file for eth0. To anything to your disk image you must first mount it:

    # mount -t ext2 -o loop /u01/xen/xen1-rootfs /mnt/tmp
    

    and copy across a few useful files:

    # cp /etc/{shadow,group,passwd,resolv.conf} /mnt/tmp/etc
    

    I also created a /etc/fstab file inside my disk image which looked like this:

    /dev/sda1       /       ext3    errors=remount-ro       0       1
    /dev/sda2       none    swap    sw                      0       0
    proc            /proc   proc    defaults                0       0
    

    Also create yourself a ifcfg-eth0 file so that networking is brought up on startup, mine looks like the one below change the ip address etc to suit:

    # cat /mnt/tmp/etc/sysconfig/network/ifcfg-eth0
    BOOTPROTO='static'
    BROADCAST='192.168.254.255'
    IPADDR='192.168.254.52'
    MTU=''
    NAME='Ethernet Network Card'
    NETMASK='255.255.255.0'
    NETWORK='192.168.254.0'
    REMOTE_IPADDR=''
    STARTMODE='auto'
    USERCONTROL='no'
    _nm_name='static-0'
    

    When that is done, umount the disk image (remember to cd out of any directory below /mnt/tmp first):

    # umount /mnt/tmp
    

    Now you are ready to start your Xen virtual domain:

    # xm create xen1 -c
    

    If all goes well you should see what looks like a kernel booting and it will drop you at a login prompt, because we copied over your local passwd and shadow files you should be able to just login. There are some services which need turning off because they dont play well inside the VM (like the HW clock etc) but you can simply disable these by running yast when your virtual machine has booted.

    There is a whole lot of extra stuff you can do with xen but all of that is outside scope of this document, I wanted to keep this as a short ‘How I made it work on Suse 9.3’ for more information check the documentation linked from the Xen home page at http://www.cl.cam.ac.uk/Research/SRG/netos/xen/


  • Mirroring SuSE

    I find myself having to mirror bits of the SuSE install tree and updates quite often. The problem is I typically do not need all the .iso files, the src rpms and the debuginfo packages, so I use the command below to exclude them from my rsync:

    *.iso
    *-debuginfo-*
    */src/*
    *-ar-*
    *-ca-*
    *-cs-*
    *-da-*
    *-de-*
    *-el-*
    *-es-*
    *-et-*
    *-fi-*
    *-fr-*
    *-hu-*
    *-it-*
    *-ja-*
    *-ko-*
    *-nl-*
    *-pl-*
    *-pt-*
    *-ru-*
    *-sk-*
    *-sl-*
    *-sv-*
    *-tr-*
    *-zh-*
    *-af-*
    *-bg-*
    *-br-*
    *-cy-*
    *-en-*
    *-eo-*
    *-fo-*
    *-ga-*
    *-hr-*
    *-is-*
    *-nn-*
    *-ro-*
    *-nb-*
    

    **hint:**You can paste the above list into a text file and pass it to rsync with the --exclude-from option

    Also note that from the UK, I’ve found ftp.sh.cvut.cz to be a consistently fast mirror


  • Hacking the SUSE boot.iso

    I’ve recently been using autoyast quite a bit to automate builds of new SUSE this is nice and easy if the machine network boots, you just use pxelinux but if the machine does not support PXE booting you have to boot from the mini boot.iso included with SUSE and then manually type in the install= and autoyast= kernel parameters.

    To save time and the hassle of doing this I have built a custom SUSE 9.2 mini boot.iso with a new grub option which includes all of the information necessary. Below is the steps involved:

    1. get the mini boot iso from a SUSE mirror, the filename I grabbed was /pub/suse/i386/9.2/iso/SUSE-Linux-9.2-mini-installation.iso
    2. mount the iso: ```

    mount -t iso9660 -o loop SUSE-Linux-9.2-mini-installation.iso /mnt/tmp

    
      3. copy the contents to a temporary directory and set them to be writeable: 
        ```
    # mkdir /tmp/boot-iso
    # cp -av /mnt/tmp/* /tmp/boot-iso/
    # chmod -R 700 /tmp/boot-iso/*
    
    1. edit the boot loader config in boot/loader/isolinux.cfg. I added the following as the second option: ``` label cmlinux kernel linux append initrd=initrd ramdisk_size=65536 splash=silent showopts
      textmode=1 install=http://192.168.22.250/install/SUSE/i386/9.2/
      autoyast=http://192.168.22.250/install/SUSE/autoinstall/repository/
      textmode=1
        
        **note**: the lines are broken here for clarity make sure the append line is all on one line and the slashes are removed
        
        **note**: the url's specified in the `install=` and `autoyast=` parameters are specific to our configuration, you will need to edit as appropriate for your own auto install setup
        
        you also need to edit the messages file which is what is displayed on screen immediately after boot, I added the following directly after the 'Boot from harddisk' option:
        
        ```
    cmlinux   - Use the CM autoyast installer
    
    1. now you are ready to make your bootable iso. Whilst inside the /tmp/boot-iso/ directory I ran the following: ``` #mkisofs -R -b boot/loader/isolinux.bin -no-emul-boot
      -boot-load-size 4 -boot-info-table -o ../custom-boot.iso .
    
      6. write this `custom-boot.iso` to a cd and boot from it. When you see the menu choose 'cmlinux' which will automatically set the install source to be the url in the `install=` parameter and set the autoyast repository to the url in the `autoyast=` parameter. If there is an entry in the rules.xml for the machine being booted an auto install should run automatically, if not you will be left with the option of doing a manual install