I'm having the same problem on Ubuntu 11.04 64b with Gnome 2 and the proprietary nVidia drivers for a nVidia 9600 GT. I've also tried the latest nVidia and X versions from the xorg-edgers PPA, but to no avail. The Xorg process starts at ~50MB and after a while it randomly jumps to 500-700MB in a matter of minutes. There doesn't seem to be a certain program that causes it. Closing all of my programs doesn't solve anything, but a log-out does (I'm guessing it restarts the X server). I'll be happy to post any logs if someone is interested.
I'm having the same problem on Ubuntu 11.04 64b with Gnome 2 and the proprietary nVidia drivers for a nVidia 9600 GT. I've also tried the latest nVidia and X versions from the xorg-edgers PPA, but to no avail. The Xorg process starts at ~50MB and after a while it randomly jumps to 500-700MB in a matter of minutes. There doesn't seem to be a certain program that causes it. Closing all of my programs doesn't solve anything, but a log-out does (I'm guessing it restarts the X server). I'll be happy to post any logs if someone is interested.