nVidia 3D desktop effects with XEN on SLED 10 SP2

Here's my first post for the new year 2009...

Having 3D desktop effects (ie Compiz) on SUSE Linux Enterprise Desktop (SLED) 10 is not a new thing since its been available for a few years. However, if you are using Xen virtualization with SLED, you would have realized that the Xen kernel don't play nice with these fancy graphics.

About a year ago, some brilliant and kind soul published how he got his Thinkpad T61p with nVidia graphics card to work with Xen and also enabled the 3D desktop effects. An awesome article (link here).

However, the technique used required a specific patch to be applied to a specific nVidia driver. About a month or so ago, I came across another entry online that talks about using the latest (and greatest) nVidia driver with Xen ... WITHOUT patching the nVidia driver!

Here are the summarized steps (link and credit to muchologo's original entry on nvnews forum):

Pre-reqs: Ensure you have installed kernel-source and C/C++ tools.

1. Download the desired nVidia driver. Link to good nVidia driver info.
2. Install the nVidia driver per instructions on a non-Xen SLED 10 and check that 3D desktop effects works. Link to custom install of nVidia driver on SUSE.
   
3. Reboot into the Xen SLED 10. Basic desktop GUI should fail to work (that's normal).
4. Prepare the kernel:
• cd /usr/src/linux
• cp arch/x86_64/defconfig.xen .config
• If you are using 32bit kernel, change x86_64 to i386
  
• make oldconfig && make scripts && make prepare
5. At the default TTY1, login as root, expand the nVidia driver package via:
• ./NVIDIA-Linux-x86_64-177.82-pkg2.run --extract-only
• You could be using a newer driver/arch, the above is just a sample using version 177.82 and x86_64 arch.
6. Here's the magic:
• IGNORE_XEN_PRESENCE=y CC="gcc -DNV_VMAP_4_PRESENT -DNV_SIGNAL_STRUCT_RLIM" make SYSSRC=/usr/src/linux module
• This will compile a new nvidia.ko module that will work with Xen
7. Now, replace the newly compiled nvidia.ko file and restart the nvidia module as follows:
• cp nvidia.ko /lib/modules/-xen/kernel/drivers/video/
• cd /lib/modules/-xen/kernel/drivers/video/
• depmod -a
• modprobe nvidia
• startx
8. All the steps only needs to be performed once for each new driver or kernel update.

Have fun!