Tuesday, April 7, 2009

nVidia 3D desktop effects with Xen on SLED 11

YES! I did it! I've moved all my machines from SUSE Linux Enterprise 10 SP2 to SUSE Linux Enterprise 11. senyum As you can see from the screenshot above, 3D effects (Desktop Cube) with Windows XP running as a virtual guest on Xen on the right.

Those who knows me understands that this is not just about popping in a DVD and installing a vanilla OS. To be truly productive in my environment, I need to be sure (ie test, test, test) this new SLE 11 can meet and/or exceed my expectations on function/feature and interoperability with my essential applications.

So here is my first entry on how I customized my SUSE Linux Enterprise Desktop 11. I need desktop applications and eye-candy (3D effects) as well as the Xen virtualization engine. On my Thinkpad T61p, I have an nVidia Quadro FX570M. I want to be able to boot into a Xen kernel and still have the wonderful 3D desktop effects.

Pre-Reqs: Ensure you have installed the C/C++ Compiler and Tools installation pattern and the kernel-source package.

1) Download the latest nVidia Linux driver. At the time of this writing, its version 180.44. To find out which is the latest driver, check out this sticky thread on nvnew.net. Alternatively, you can go directly to nvidia.com, provide information on your card and have it re-direct you to the right driver via this link.

Note 1: Since this is a proprietary driver, obviously it would not be packaged with the default SLED 11. Nonetheless, upon vanilla install, the open source nv driver did a good job and was able to display high-res (1920x1200). Unfortunately, I did not have as much luck enabling the 3D desktop effects.

Note 2: The steps below is considered a manual/hard way of installing nVidia drivers. The advantage is you can use the latest (greatest?) driver from nVidia immediately. However, the down-side is you will need to recompile this driver every time you upgrade your Linux kernel (all other updates are ok). For an easier way, also known as the repository way where we use the nVidia Online Repository via YaST, to download and automatically install pre-compiled drivers. Please refer to these online docs [here and here] for a general idea between these 2 install options.

2) With SLED 11 (default, non-Xen kernel) running, switch to TTY1 (Ctrl-Alt-F1). Login as root and switch to runlevel 3 via the command <init 3>. Execute the NVIDIA driver you have downloaded. You might need to make that driver executable first via <chmod +x NVIDIA-Linux-x86_64-180.44-pkg2.run > and execute it < ./NVIDIA-Linux-x86_64-180.44-pkg2.run >

The driver installation program is rather straight-forward (most of the time just hitting Enter and choosing Yes). The program will inform you it cannot find any pre-compiled driver, just hit Enter and it will go ahead and compile the driver for SLED 11. If you are running a 64-bit SLED 11, it will offer to install 32-bit compatible "stuff", just accept it with Yes. Finally, it will offer to edit your configuration files, accept it with Yes. Easy. kenyit

Finally, edit the /etc/X11/xorg.conf file and change "Off" to "On" in the following line:

Section "Extensions"
Option "Composite" "on"

3) Test that the nVidia driver works by going back to runlevel 5 via the command <init 5>. You should get full GUI. Log in and enable the 3D desktop effects Computer -> Control Center -> Look & Feel -> Desktop Effect. Check the Enable desktop effects checkbox.

Tip: After clicking on the checkbox, a dialog box asking if you want to keep the new settings appear. WAIT! DO NOT click Yes for another 3-5 seconds. This allows time for the 3D engine to initialize and should anything go wrong, it will revert back to 2D in 30 seconds automatically. Only when you see 3D effects (ie try moving the opened windows around), then click Yes on the dialog box to keep these settings.

4) As root, copy the compiled nvidia driver to another location and rename it for safe keeping. This is important because you will need to run the nVidia installer program again against the Xen kernel. This process will over-ride and remove this previously compiled driver for the default kernel.

As root and in your root directory (/root), execute the following command:
< cp /lib/modules/ ./nvidia.ko.default >

5) Reboot SLED 11 into the Xen kernel. You should fail to enter runlevel 5 (Full GUI) as the required nVidia driver for Xen has not been compiled yet. Log in as root on TTY1 and Re-run the nVidia installation program but do it via this "magic" line:


The nVidia driver installation program will run again (see step 2) and this time, it will compile the driver against the Xen kernel. Highly recommend you take a copy of the newly compile nVidia driver:

< cp /lib/modules/ ./nvidia.ko.xen >

Once successfully completed, go into runlevel 5 and voila!

6) To ensure that you can have full desktop GUI and 3D effects on both SLED 11 Default and Xen kernel, verify and copy the correct <nvidia.ko> files previously backed up (nvidia.ko.default and nvidia.ko.xen) respectively to:

(nvidia.ko.default) -> /lib/modules/
(nvidia.ko.xen) -> /lib/modules/

Done and done! senyum


  1. I got this on xen mode install start:
    ERROR: unable to determine the NVIDIA kernel module filename.

    ake:*** [print-module-filename] Error 1

  2. Hi Shuker,

    I've never seen that error before... based on that error message, I would check if there are other nVidia modules that are already installed (ie the rpm ones).

    > rpm -qa | grep -i nvidia

    If the above returns something you've got to delete it via rpm -e command.

    Hope this helps.

  3. Hi, i got working nvidia amd 3D effects with my pae kernel. Do you mean that i should delete it and build it first for the xen kernel and then for pae?

  4. Hi Shuker,

    Ahh, I think I understand now. Yes, everytime you run the command in step 5, it will generate the nvidia.ko file for that kernel you're running.

    Yes, you'll need to re-run step 5 in the xen kernel for the right nvidia.ko generated for xen kernel.

    If you intend to reboot back into the pae kernel in the future, suggest you make a backup of the nvidia.ko file first so you do not need to run the program in step 5 everytime... instead, just replace the nvidia.ko with the backed up one... hope this makes sense.

    Cheers and good luck!

  5. Hi

    I am having the same problem, I am running Opensuse 11.2 64 bit with Nvidia GetForce 9600 GT.

    First I installed the Opensuse 11.2 and all good but I cant enable the compiz so I decided to to go for the propriety Nvidia driver, they released 2 weeks ago, I installed and the compiz works fine but once I try to boot from xen mode, it goes to terminal tty and the X server wont start.

    Then I added your magic line, it start to build the kernel and it will stop and I got the following error:

    ERROR: unable to determine the NVIDIA kernel module filename.

    I am not running any other nvida driver.

    any reply will be great, thanks for your post

  6. Hi Sarhang,

    1) You managed to successfully install the nVidia driver and have 3D with the default kernel in openSUSE 11.2. Yes?

    2) You want the same when using the Xen kernel. Using the "magic" line in step 5, your attempt to compile the nVidia driver against the Xen kernel failed. Yes?

    If so, here's my suggestion:

    a) When booting into Xen, its normal to encounter the terminal as the X server won't start. That's why you need to run that "magic" line to compile the driver. But you only need to run it once successfully... and of course if you upgrade your Xen kernel to a later version.

    b) Since you encountered the error where the installer is confused (unable to determine) the nVidia modules installed. You might have more than 1 nVidia driver installed... I noticed that YaST sometimes installs the default nvidia drivers from online repositories when you attempt to upgrade or install other modules that have this dependency... another long story...

    c) So, do a rpm -qa | grep -i nvidia and see if you have multiple entries. If so, remove all of them. Yes, since you're installing nVidia drivers manually, it should not appear in RPM. To play it safe, you might also want to disable the nVidia repository in YaST (yast2 repositories).

    Hope this helps. Cheers!

  7. Hi

    Thanks for your reply, look I went back to SLED11, I installed the HP driver for Nvidia for SLED 11, everything works fine, the Compiz works fine but still in the XEN mode the X11 wont start.

    In the step four, when I try to copy the default kernel to the Xen kernl I dont have:

    cp /lib/modules/ ./nvidia.ko.default

    but I have:


    cp /lib/modules/ /lib/modules/

    is that ok.

    I will let you about the Opensuse 11.2 later

    Many thanks

  8. sarhang,

    1) Not sure what you mean by "HP driver for Nvidia". I use nVidia drivers from nvidia.com

    2) The nVidia installer (when successfully executed) will generate nvidia.ko in /lib/modules/[kernel version]-default/kernel/drivers/video. Do not replace nvidiafb.ko!

    3) The [cp] step is meant to backup nvidia.ko for the default kernel.

    4) You are supposed to reboot into the Xen kernel, experience tty only, and re-run the nVidia installer with the magic line. This will generate another nvidia.ko but in the /lib/modules/[kernel version]-xen/kernel/drivers/video/ directory. This is totally different directory in point 2 above.

    5) If (4) is success, execute init 3 followed by init 5 and you should have full GUI for Xen.

    6) When you boot back to default kernel, you will experience tty again because the last nVidia installer (executed in Xen) has removed the nvidia.ko for the default kernel. This is when you can cp the previously backed up nvidia.ko back into the appropriate directory.

    Hope this clarifies it... its hard to explain it here with just words... do hope you get what I mean to write. :)

    Best of luck!

  9. Hi Kam,

    Thank you for your reply, sorry for being pain. I am running now Opensuse 11.2.
    I have the following kernel:

    rpm -qa | grep -i kernel











    I installed the Nvidia driver (VIDIA-Linux-x86_64-190.42-pkg2.run) for Geforce 9600 GT, I download from Nvidia web site,

    # lsmod | grep -i nvidia

    nvidia 9636872 38

    everything went fine, I installed it and compiz is working.

    Then I reboot the I went to XEN mode and fails and the X server did not start and went o tt1, which is fine then I tried to install the drive with your magic line:


    it will de-install the Nvidia driver which was installed in Non Xen kernel and after that it says “ERROR: Unable to build the NVIDIA kernel module.

    Here is log file: it would not let me to copy the whole the log file, but this is the end of the of the file, if you want I could email the log fiel:

    cat /var/log/nvidia-installer.log

    include2/asm/bitops.h:64: warning: pointer of type ‘void *’ used in arit


    include2/asm/bitops.h: In function ‘clear_bit’:

    include2/asm/bitops.h:102: warning: pointer of type ‘void *’ used in ari


    include2/asm/bitops.h: In function ‘change_bit’:

    include2/asm/bitops.h:178: warning: pointer of type ‘void *’ used in ari


    In file included from /usr/src/linux-,

    from /usr/src/linux-


    from /usr/src/linux-


    from /usr/src/linux-

    make -f /usr/src/linux-

    scripts/mod/modpost -m -a -i /usr/src/linux-

    dule.symvers -I /tmp/selfgz3614/NVIDIA-Linux-x86_64-190.42-pkg2/usr/src/nv/M

    odule.symvers -o /tmp/selfgz3614/NVIDIA-Linux-x86_64-190.42-pkg2/usr/src/nv

    /Module.symvers -S -K /usr/src/linux-

    ers -M /tmp/selfgz3614/NVIDIA-Linux-x86_64-190.42-pkg2/usr/src/nv/Module.mar

    kers -w -s

    WARNING: could not find /tmp/selfgz3614/NVIDIA-Linux-x86_64-190.42-pkg2/usr/

    src/nv/.nv-kernel.o.cmd for /tmp/selfgz3614/NVIDIA-Linux-x86_64-190.42-pkg2/


    FATAL: modpost: GPL-incompatible module nvidia.ko uses GPL-only symbol 'xen_


    make[4]: *** [__modpost] Error 1

    make[3]: *** [modules] Error 2

    make[2]: *** [sub-make] Error 2

    NVIDIA: left KBUILD.

    nvidia.ko failed to build!

    make[1]: *** [module] Error 1

    make: *** [module] Error 2

    -> Error.

    ERROR: Unable to build the NVIDIA kernel module.

    ERROR: Installation has failed. Please see the file

    '/var/log/nvidia-installer.log' for details. You may find suggestions

    on fixing installation problems in the README available on the Linux

    driver download page at www.nvidia.com.

    it is been very frustrating and I cant work out how to have XEN + Compiz, it is pain.

  10. Hi

    Could you tell me what I am doing wrong ?
    Many thanks


  11. Hi Sarhang,

    Sorry, I was traveling on assignment. I've installed openSUSE 11.2 on another partition on my Thinkpad T61p with nVidia Quadro FX570M.

    It appears that my documented hack does NOT work with openSUSE 11.2 - 2.6.31 kernel. It works for SLED 11 - 2.6.27 kernel. Sorry :( This is beyond me.

    Looks like we'll have to wait for someone to come up with new howto hacking this new generation of kernel using Kconfig.

  12. Hi Kam,

    Thank you for your reply, I opened service request with HP, I will let you know about the outcome, thanks again for your time.