Xen Bridged networking with NIC Bonding on SLES

We have a customer that uses the SLES Xen hypervisor (SLES 10 SP2 and now SLES 11) on their test and development server. The idea is simple, virtualize all their test and development servers on a single 2-way QUAD core, 32 GB RAM and 1TB RAID-5 SATA drive. This way, they save money on the number of physical machines in their test/development environment, improve utilization (not all projects runs concurrently). Their production systems are still running on bare-metal without virtualization... for now.

They've been having some networking issues (ie packet drop in PING test) and suspected its either the NIC (Broadcom 4 port Gb NIC and using NIC bonding) hardware or the netorking setup needs tweaking. Here's a journal of what I discovered and resolved (to a certain extent - hardware not in my scope):

1. Boot up SLES 11 (x86_64) with the default kernel (non-Xen) and configure the basic networking. Created a bond0 device that bonds 2, out of the 4, physical ports for NIC bonding (static IP, DNS and Routing info provided by customer). Tested the configuration and it works, we can ping other servers and development desktops can ping the server. Opened port in SuSEFirewall and SSH session works. Used yast2 lan for configuring and verified (always a good thing) via the following files /etc/hosts, /etc/resolv.conf, /etc/sysconfig/network/ifcfg-bond0

The following steps are done with reference to this Novell support document "Hassle-free Xen Networking" at this link.

2. Restart SLES 11 with the Xen kernel. Verify the following entry is commented out in /etc/xen/xend-config.sxp

## (network-script network-bridge)

and instead have the following:

(network-script )

Restart Xen via rcxend restart.

3. Created a bridge interface called br0 and bridged it to bond0. Moved all the static IP settings from bond0 to br0. In the place of the now empty bond0 (zero IP settings), I placed static IP 0.0.0.0 and Subnet Mask /32 (or 255.255.255.255). Verify that br0 is the interface with the static IP while bond0 does not. Also verify all the network pings between the servers and development desktop.

You can see this in more detail as referenced earlier at this link.

4. Almost there! Next, I just needed to verify and update virtual NIC settings in all domUs (VMs). By inspecting each VM configuration file in /etc/xen/vm/, we need to amend the vif variable/s (see example below):

from: vif=[ 'mac=00:16:3e:24:96:38', ]
to: vif=[ 'mac=00:16:3e:24:96:38,bridge=br0', ]

Don't forget to refresh the configuration via xm delete [VM] and xm new [VM] where [VM] is the name of the domU (VM).

Done! Customer can ping the host (SLES 11) and their domUs (SLES, RHEL and Windows) successfully without any packet drops.

The SLE 11 Community Preview in Singapore

The SUSE Linux Enterprise 11 community preview in Singapore was held last Wednesday evening (22nd April, 7pm). Well... we really got started at 7:30pm as we were chatting away and waiting for some of the regulars to arrive. Darrel and Mike did a fantastic job organizing and getting the word out. Thank you!

I'm looking forward to visiting the next community SUSE meetup in May and also supporting Darrel in Singapore's Software Freedom Day in September.

We had slightly over 35 people turn up and spent slightly over an hour (initially planned 45minutes) on my presentation and demo. I was delighted at the questions and discussions, a real lively session. This was followed by some light snacks sponsored by Novell.

Update:
Link to Meetup -> http://www.meetup.com/sg-linux/calendar/10093898/
Link to Photos -> http://www.meetup.com/sg-linux/photos/596475/
Link to Presentation -> http://files.meetup.com/95900/SLE-11-CommunityLaunch-SGP.pdf

3G Surfing with SLED 11

YES, it works! Out of the box babe!

With reference to my previous entry last year and compared to getting my Telco's(StarHub) 3G mobile broadband modem(Huawei E270) to work [link here], life is so much better in SLED 11. Here are the really simple steps:

1) Plug in my 3G modem(Huawei E270) via USB.
2) Wait 5 seconds and my NetworkManager applet on my desktop picks it up. Select Properties under the device shown and do a 1-time setup (eg click the Connect Automatically checkbox).
3) Wait for the connection and DONE! Its that simple... so simple that I wondered if I should blog about it. Hahaha .

Life is good.

External Projector or Screens with SLED 11, nVidia and Thinkpad T61p

Having setup my SUSE Linux Enterprise Desktop 11 on Thinkpad T61p working with Xen previously [link here], I now needed to project my screen (1920x1200) onto an external projector (typically 1024x768). Additionally, I have an external LCD monitor which I would like to use as my secondary monitor (ie Multi-head, desktop spanning blah).

Good news is it WORKS!

Here are my observations and the neat little trick (tool) that worked for me:

Note: After plugging in the VGA cable to Thinkpad T61p, always restart (re-initialized) X11 (Ctrl-Alt-Backspace) before doing anything else. On SLED 11, you will need to hit Ctrl-Alt-Backspace TWICE within 1-2 seconds. I think this is a newly introduced safety feature to avoid accidentally hitting this key combination and killing the entire X11.

1) SAX2 in SLED 11 does not seem to drive the nVidia driver like it used to do in SLED 10, especially the part where we configure Dual Head Mode. Irregardless of how I tried to configure the Dual Head Mode settings in SAX2, when I connect to an external projector or LCD monitor and after re-initializing X11, SLED 11 will project itself onto the external display and my Thinkpad LCD will be blank.

2) The trick to get the display to show on both the Thinkpad LCD (1920x1200) and external screens in the correct resolution is to use the nVidia tool that comes with the driver. As root, you startl it up via nvidia-settings. Below are 2 screenshots of it in action. I run it as root so that any changes I made can be written to the xorg.conf file directly.



If I had an external screen connected to my Thinkpad T61p as I'm writing this, the second screenshot would have 2 screens set side-by-side for me to configure the resolution, position and designate the primary display when using the TwinView mode.

Of course, the SLED 11 3D desktop effects continue to work across screens. Very sweet... so sweet that my good friend, a loyal Windows Vista user and graphics guru, is impressed.