Configuring and Troubleshooting X

Last modified: July 16, 1999
Development stage: Beta

This section is guaranteed to be long, so if you want to read the whole thing, it's your choice. This includes just some basic things for Red Hat 5.2, based on XFree86 version This information should also work for other Linux distributions.

I'm assuming you already have XFree86 installed, along with its libraries. If not, install them. In Red Hat, TurboLinux, Mandrake, SuSE, and Stampede, they come in RPM files that start with "XFree86". If you want to install the files from the tarred and gzipped form, read Upgrading and Installing X.

  1. What's an X Server?
  2. Configuring X
  3. A Common Error Message
  4. The Infamous Gray Screen
  5. Alias to Start X
  6. Turning Off Virtual Desktop
  7. Additional Fonts
  8. Mouse Trouble
  9. Newer Cards

What's an X Server?

An X server is sort of like a video driver (that's the best I can explain it...) that the system uses to run the graphical user interface. There are too many changing X servers to list here, but there are SVGA, VGA, and Monochrome, which are the most basic. There are also commercial X servers which I know virtually nothing about.

The X server you choose to use depends on the type of chip your video card uses to process graphics. It also determines how many colors will be displayed in X (the amount of video memory you have also determines that). When you key in startx and the error under "A Common Error Message" appears, you should make sure that you installed the server that you configured for.

The X server I used was the S3V. That stands for S3 ViRGE, since my Diamond Stealth 3D 2000 used that chip. When I got a Creative Labs Graphics Blaster Riva TNT (PCI), I switched to the SVGA server because the SVGA server was the one that had support for the Riva TNT chipset.

Configuring X

If there is any kind of error given when you execute the startx script, you probably should run Xconfigurator or xf86config. These will configure the keyboard, the mouse, the video card, the monitor, and some other stuff so that X will run properly. The settings are stored in a file called XF86Config. On my system, it's in /etc/X11/XF86Config. You can edit that with a text editor such as pico, vi, joe, jed, or emacs.

A lot of people have trouble configuring X, especially beginners and long-time text mode users. The best thing to do when there are no manuals available is to guess. It might take a few hundred tries, but using defaults will usually get you by. For example, I didn't know which RAMDAC and Clockchip setting to use when eventually I read the little comments/tips on the screen and decided to try not using a RAMDAC or Clockchip setting for my card. It turns out that I wasn't supposed to use those settings in the first place.

It will help you greatly if you know your hardware settings or have your hardware manuals handy. If not, experiment.

A Common Error Message

The following error message is very common among new users, and often very confusing as well:

_X11TransSocketUNIXConnect: Can't connect: errno = 111
giving up.
xinit:  Connection refused (errno 111):  unable to connect to X server
xinit:  No such process (errno 3):  Server error.

Solution One

One solution to this is to install the server you chose for your video card's chipset (the type and brand of chip it uses to process video information). In Red Hat, you install the RPM package that describes which server it is by the first few characters in its filename. The common X servers are as following, with their RPM package filenames in parentheses:

To avoid as much trouble as possible, install these RPM packages if you're using Red Hat.

  1. XFree86-3.3.2-8.i386.rpm - XFree86
  2. XFree86-devel-3.3.2-8.i386.rpm - Lets you compile stuff
  3. XFree86-libs-3.3.2-8.i386.rpm - X libraries
  4. XFree86-SVGA-3.3.2-8.i386.rpm - 256 color X server
  5. XFree86-100dpi-fonts-3.3.2-8.i386.rpm - Fonts
  6. XFree86-75dpi-fonts-3.3.2-8.i386.rpm - Fonts
  7. Xconfigurator-3.57-2.i386.rpm - Helps with configuring X

For more information on installing and upgrading X, you might want to read Installing and Upgrading X.

If you want to run at more than 256 colors use another X server that is made especially for the type of chip that's on your video card. The special X servers are made for the newer PCI video cards, but ISA graphics accelerator cards could possibly also use the special X servers.

Solution Two

Another possible cause for the "_X11TransSocketUNIXConnect" error is that the mode that is chosen (or the mode that you set it to) is not valid. This can be because of monitor settings that are too high or too low for your video card. I discovered this when I set my monitor's hsync (horizontal sync) and vsync (vertical sync) ranges to settings that were too high, although I had the proper X server for my card. Then I tried them really low, and then I was allowed into X--but the resolution was very low. I finally decided on something in the middle and that seemed to work.

Those monitor settings (hsync and vsync ranges I think) will also have an effect on what resolution you can run at. Again, setting those too low will get X to make you run at a very low resolution, so that everything looks huge and blocky. Try entering your own ranges if your monitor manual has the settings. You can also select what type of monitor you have from XFree86's list of monitors. My MAG Innovision DX15T was not listed but something close was, the DX1595, and it had the exact hsync and vsync ranges that my monitor had (when I finally found it in the manual).

Otherwise, you can try the lowest setting and work your way up, try the highest setting and work your way down, or choose something in the middle and see if your monitor supports it. Be careful: if the setting you choose is too high, you could seriously damage your monitor.

The Infamous Gray Screen

Many people encounter the problem of starting X (startx), seeing an X-shaped mouse-pointer-thingy, a gray screen, and being taken back to text mode with no window manager loading and no error messages. If it doesn't take you back to text mode, try the Ctrl-Alt-[Backspace] combination.

The way I found I could solve this was to create a .xinitrc in my home directory and putting the line exec afterstep in it. That seemed to work fine and it loaded the AfterStep window manager. As you can probably figure out, you'll have to replace afterstep with whatever window manager you're using. If I wanted to use fvwm, I'd put in exec fvwm instead.

Now suppose you want to run multiple window managers; of course, you can run only one at a time in each X session, but you don't want to type everything in again. The solution is to type in the startup lines of each window manager then comment out all the ones that you don't want to use, leaving one line uncommented so that it'll run. Here's how my .xinitrc file looks like:

# exec afterstep
# exec fvwm95-2
if [ -f $HOME/GNUstep/Library/WindowMaker/.workspace_state ]; then
	sh $HOME/GNUstep/Library/WindowMaker/.workspace_state &
exec wmaker

That is set to run WindowMaker when I start up, and the commented lines, the ones with the pound signs (#) at the beginning of the lines, are ignored. If you leave everything commented out, it'll give you the gray screen with the X-shaped mouse, so if that happens, you should know what to look for. :)

Alias to Start X

On my system, I start X by typing x at the prompt, and it starts up in 24 bits per pixel mode when I do that. I can type startx -- -bpp 24, but that takes time to type up. I can do this because I added a line to /etc/bashrc that goes like this:

alias x='startx -- -bpp 24'

Of course, if you prefer to run at some other video mode other than 24 bits per pixel (16.7 million colors, I think) then you would replace "24" with 15, 16, 32, or whatever you may wish to run as the default.

With Slackware, you might have to edit /etc/profile instead of /etc/bashrc if it doesn't look in /etc/bashrc when a user logs in. I don't think aliases are supposed to go in /etc/profile, but it works.

Turning Off Virtual Desktop

XFree86 usually annoys new users with something called virtual desktop. This can be described as a screen bigger than the one displayed, so that when a user moves the mouse to the edge of the screen, it scrolls down instead of staying there.

Only after using Linux for a year did I discover how to turn that off. Yeah, it gives me more desktop space, but as new users frequently complain, it's annoying. :)

Anyway, the way I turned it off was by finally finding the right place to edit in the /etc/X11/XF86Config file. The following is the area that you should edit:

Section "Screen"
    Driver      "accel"
    Device      "Diamond Stealth 3D 2000"
    Monitor     "My Monitor"
    Subsection "Display"
        Depth       8
        Modes       "800x600"
        ViewPort    0 0
        Virtual     800 600
    Subsection "Display"
        Depth       16
        Modes       "800x600"
        ViewPort    0 0
        Virtual     800 600
    Subsection "Display"
        Depth       24
        Modes       "800x600" "640x480"
        ViewPort    0 0
        Virtual     800 600
    Subsection "Display"
        Depth       32
        Modes       "800x600" "640x480"
        ViewPort    0 0
        Virtual     800 600

This can be found towards the end of the file. At first I heard of people editing this file and successfully getting the results that they wanted; I tried it, but I edited the wrong part of the file and that didn't make a difference at all.

Anyway, the way I got rid of my virtual desktop was to set it to be the same resolution as the resolution I was running at. In my case that was 800 by 600 (all modes from 8 bpp to 32 bpp). You can also try commenting out the lines starting with "Virtual" by putting a pound sign (#) at the beginning of the line.

Additional Fonts

Most people might not be satisfied with the fonts included with the standard distribution of X, so if you're one of them, read Zach Beane's tutorial on adding fonts to X.

The fonts themselves are available from They're scalable Adobe Type1 fonts, which means that they won't get jagged when enlarged to large font sizes.

You can also get a couple of packages containing fonts from They're the .tar.gz files.

With Zach Beane's tutorial, it's also possible to add other types of fonts, such as 75dpi and 100dpi.

Mouse Trouble

There are a lot of possible problems people might encounter with mice. Just make sure you have it pointing to the correct device (usually /dev/ttyS0, which is COM1 under DOS and Windows) when the X configuration utility that you use (xf86config, Xconfigurator, etc.) asks you where your mouse is. If you have a PS/2 mouse, use /dev/psaux.

If you want to edit /etc/X11/XF86Config, look under the section "Pointer". Here's what it looks like:

Section "Pointer"
    Protocol    "Microsoft"
    Device      "/dev/ttyS0"

If it's on another device, such as COM2 under DOS, it would be /dev/ttyS1. COM3 under DOS/Windows would then be /dev/ttyS2, and so on.

andres salomon wrote in to say that if your PS/2 mouse doesn't work in X but works in normal text mode, you should try killing the GPM mouse services (killall gpm as root should do the trick). He also mentioned a workaround, which he didn't successfully use. It's adding /dev/gpmdata to /etc/X11/XF86Config, supposedly, and starting GPM with the "-R" flag.

Fotis Karpenissiotis wrote, "I had several problems using a Microsoft Mouse (Serial Mouse 2.0A) under XFree86 3.3.2. (Under 3.3.1 it worked fine). The solution I found was to start gpm using:

gpm -t ms -R

and the following lines in XF86Config:

Section "Pointer"
       Protocol    "MouseSystems"
       Device      "/dev/gpmdata"

That's about all the information I can give about mouse trouble and their solutions right now, but if you've got a different problem and a different solution then please let me know.

Newer Cards

Dragoon and Larry Elmore wrote in with tips reminding those of us with newer cards to head over to SuSE creates a Linux distribution, but they also create X servers for video cards/chipsets that XFree86 has not included yet. The XFree86 Project only releases new X servers every six months, so in that time span in between, SuSE releases stuff. It's worth checking out if you're not having much luck with that new video card and X.

If you have anything from NVidia, find their Open Source XFree86 patches/modifications on their website at This includes support for cards with Riva TNT and Riva TNT2.

If you can't find an X server for your card using any of the above methods, just about the last thing you can try is going to your card or chipset maker's website. Unfortunately, some video card companies do not like to cooperate with the XFree86 project.

Send any and all feeback to joshuago at users dot sourceforge dot net. Feedback is welcome and highly encouraged. You can also make use of the help form or guestbook, which is really more like a feedback form.

Copyright © 1997-1998 Joshua Go (joshuago at users dot sourceforge dot net). All rights reserved. Permission to use, distribute, and copy this document is hereby granted. You may modify this document as long as credit to me is given.