bad default swappiness for desktop systems

/etc/sysctl.conf belongs to procps, so that's presumably where the change would have to be made.

The difficulty with that is that swappiness=10 is a good setting for desktop use, but the current default of 60 is better for servers. In other words swappiness needs to be tweaked depending on which flavour of Ubuntu is being installed. That's why I didn't report it as a bug in procps as such.

This is definitely a bad bug. It renders Ubuntu almost unusable on computers with low RAM.

By contrast: with a swappiness of 10, I was even able to make an old P4 with 256 MB RAM, run acceptably well on Ubuntu 9.10 (I did the installation by means of the Alternate CD).

Please fix this bug before Lucid ships. This bug shouldn't be present in an LTS....

We don't override kernel defaults in userspace like this; if the default should be changed, it should be changed inside the kernel

So what would an appropriate default be in the kernel then? Presumably the reason the default is set where it is is because it is appropriate for some uses, i.e. dedicated servers. The problem is that there is no one default value that will suit all needs. Ubuntu already ships a patched kernel, so why should there by a problem with tweaking this one parameter? In any case you don't even need to patch the kernel. You just need an init script to do the job. What's the worst that could happen? Some servers run suboptimally? A little caution can be a good thing, but not when it means you choose to make the whole system unusable rather than implement a trivial fix.

Please don't change a status of a bug away from Won't Fix - that's not your decision, that's a decision for a developer.

If there's no appropriate default for the kernel, then there also isn't an appropriate default for the procps package.

@ Scott James Remnant: this means that Ubuntu will remain almost unusable on systems with low RAM (384 MB and less) and slow on systems with 512 MB RAM. That's a pity.

I respect the decision of the Ubuntu developers. However, maybe you could consider an init script lowering the swappiness, that only activates when RAM is 512 MB or less?

So basically Ubuntu is a distribution that is strongly geared towards the desktop user, yet it won't work properly for desktop use. A very simple adjustment would fix this, but this won't happen because it is against policy. That's not going to solve Bug #1 now, is it? Do you actually accept that there is a real issue here that has a significant impact on the usability of the system, or do you think we're just whining on about nothing? Have I misunderstood you, or did you really say that you think the kernel devs should change the default setting to one that is good for desktop systems and poor for servers?

The solution that Pjotr suggests is nice.
So it is only activated on systems with low RAM.
Perhaps still an option ?

You're right pjotr. I've got an Ubuntu installation as well, with 4gb ram, and 12gb swap! That's much.... It would be better to do something against this.

On Wed, 2010-03-17 at 10:24 +0000, Pjotr12345 wrote:

> @ Scott James Remnant: this means that Ubuntu will remain almost
> unusable on systems with low RAM (384 MB and less) and slow on systems
> with 512 MB RAM. That's a pity.
>
> I respect the decision of the Ubuntu developers. However, maybe you
> could consider an init script lowering the swappiness, that only
> activates when RAM is 512 MB or less?
>
Again, this would be better done in kernel code.

I'm not objecting to adjusting the swappiness value - I'm simply stating
that this is a kernel issue and should be fixed in the kernel code.

Scott
--
Scott James Remnant
<email address hidden>

Huh, sorry, I could have sworn there was a kernel task open for this - have opened one

@ Scott James Remnant: does this mean that you've asked the kernel team to look into this matter? That would be great. Thanks.

Can you provide us with a link to your conversation with the kernel team on this issue?

Am I right in assuming that if the kernel team would want to decrease the default swappiness, a swappiness decrease would only affect future kernel editions? Or would Lucid's 2.6.32 be patched as well?

On Wed, 2010-03-17 at 18:59 +0000, Pjotr12345 wrote:

> @ Scott James Remnant: does this mean that you've asked the kernel team
> to look into this matter? That would be great. Thanks.
>
> Can you provide us with a link to your conversation with the kernel team
> on this issue?
>
I have had no conversation with the kernel team.

Scott
--
Scott James Remnant
<email address hidden>

Well, why haven't you Scott? This is a serious issue and really needs to be fixed in time for 10.04 Final.

From the Ubuntu Documenatation: https://help.ubuntu.com/community/Installation/SystemRequirements#Desktop%20installation
[quote]Recommended minimum requirements

Ubuntu should run reasonably well on a computer with the following minimum hardware specification. However, features such as visual effects may not run smoothly.

    * 700 MHz x86 processor
    * 384 MB of system memory (RAM)
[...]
[/quote]
If we can't meet those requirements due to a bad swappiness-value, someone (I'm not saying who, since I don't have a clue how those things work) needs to fix it in time before the LTS is released.
Just my 2 cents.

On Wed, 2010-03-17 at 20:22 +0000, Vistaus wrote:

> Well, why haven't you Scott? This is a serious issue and really needs to
> be fixed in time for 10.04 Final.
>
Because that's not my job? :)

Scott
--
Scott James Remnant
<email address hidden>

Vistaus: by setting this bug to Confirmed, you actually prevented the kernel team from seeing this bug -- this is why this has been held up for so long

one obvious solution is that -generic should have a desktop-y swappiness, while -server should have a server-y one

Hi Robert,

If you could also please test the latest upstream kernel available that would be great. It will allow additional upstream developers to examine the issue. Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Once you've tested the upstream kernel, please remove the 'needs-upstream-testing' tag. This can be done by clicking on the yellow pencil icon next to the tag located at the bottom of the bug description and deleting the 'needs-upstream-testing' text. Please let us know your results.

Thanks in advance.

    [This is an automated message. Apologies if it has reached you inappropriately; please just reply to this message indicating so.]

This bug report was marked as Incomplete and has not had any updated comments for quite some time. As a result this bug is being closed. Please reopen if this is still an issue in the current Ubuntu release http://www.ubuntu.com/getubuntu/download . Also, please be sure to provide any requested information that may have been missing. To reopen the bug, click on the current status under the Status column and change the status back to "New". Thanks.

[This is an automated message. Apologies if it has reached you inappropriately; please just reply to this message indicating so.]

This bug is most definitely NOT expired and still present in Lucid.

Jeremy, I think your decision to close this bug was ill-judged. It is only a month since you asked me to test whether the bug continues in Lucid. What this adds up to is that I was given a month to do a distribution upgrade or the bug would not be dealt with. A distribution upgrade is a big deal and there are good reasons why sometimes people may not want to rush into it (in my case it was a combination of worry (based on experience) about breaking drivers and applications, with lack of a decent Internet connection). You could just as easily have got you answer by opening a terminal in Lucid and typing "cat /proc/sys/vm/swappiness". It would have taken you less time than it took to close the bug. Why didn't you do that? In fact when I finally did upgrade to Lucid, I chose to keep whichever configuration file I altered to get the swappiness right, so I wouldn't have been able to tell you anything useful anyway.

You should already be aware that there has been disquiet in this discussion about a perceived unwillingness to take this bug seriously, or even to acknowledge that there is a problem at all. Closing the bug so abruptly once again has not helped.

Robert,
    My decision is a result of the failure to provide the information requested in comment #20. Please review that comment and provide the necessary testing. I am completely willing to take this to the Engineers for the Ubuntu Kernel. I am not, however, willing to do so unless all information I have requested is present in the bug.

Thanks!

~JFo

Listen. For me to give you the information, because I have already tweaked my system to work round the problem, I would have to open the config file concerned (and right now I can't even remember which one it is), edit it, reboot (meaning closing down all the other stuff I am working on), test the swappiness, re-edit the config file, and reboot again, whereas all you would have to do would be to enter one command at the command line, and you can do it as a normal user so you don't even need to go to the trouble of entering your password. That would give you all the information you need to present to the kernel engineers. What is your problem? Are you actually interested in making Ubuntu work properly? Am I the only one who thinks this bureaucratic buck-passing culture is insane? It's starting to look like you think us ordinary users who make the effort to file bug reports are just troublemakers who deliberately set out to waste your time and derail the project. Please get a grip.

@Jeremy:
For me, it's a hell of a job to perform a test like you suggested in comment #20.

Furthermore, I think such a test is superfluous. For testing the current kernel in Lucid, you only have to execute this command in the terminal: cat /proc/sys/vm/swappiness

That's all. On the basis of this information you can simply ask the kernel team whether they've changed the swappiness in the newest kernels (I bet they didn't). This is a very normal question, I think. And it would save us a lot of work.

@Robert Persson: if this issue isn't being tackled, then maybe we could achieve something by contacting our respective LoCo Teams.

I know several people in the leadership of the Dutch and Belgian LoCo Teams personally. I could ask them to take this matter to the level of the MOTU's. Maybe we can coordinate this?

pjotr12345,
    MOTU will not be able to change kernel source. They will defer to the Ubuntu Kernel Team. I have already spoken with my colleagues on the team. The simple fix is, give me what I have requested and I will send the bug onward. Your LoCo teams should be able to tell you the same and the MOTU will definitely say the same.

~JFo

@Pjotr12345 What would I need to do? I'm in Belgium, so my local people would ones you already know, I think. I don't actually know what LoCo is and I can't remember what MOTU stands for, but I do want to see communication improve.

Listen again very carefully. Not only is it a lot more difficult for me than for you to get the information, the fact that I have modified whichever config file it is I modified to get round the problem means that there is a possibility that I will make a mistake when I change it back and you will actually end up getting inaccurate information. You on the other hand presumably have a clean vanilla installation of Lucid in front of you. If you simply enter the command cat /proc/sys/vm/swappiness you will be presented with the information you want with 100% certainty of its correctness. You could even present it to MOTU under the pretence that I had given the information to you rather than that you found it for yourself, if that will prevent procedural embarrassment. They will never know.

Good grief! Why do I have to talk to you like you're a little child? Can't you at least use a little bit of common sense?

I still don't think that the required testing should be done by us, but in the interest of solving this issue I did the following:

- I removed the line in sysctl.conf that set swappiness at 10 (a line that I had put in previously myself);
- reboot
- checked swappiness: it was at 60 again
- I installed the newest available kernel, kernel 2.6.35-020635rc1-generic
- reboot: failed, because of monitor "out of range"
- reboot in Recovery Mode, which also failed because of monitor "out of range"
- reboot with kernel option "nomodeset", which was succesful
- In the terminal: cat /proc/sys/vm/swappiness, which gave 60.

So the problem still exists in the newest kernel. See the attached screenshot of the terminal commands and their results.

@Jeremy Foshee: do you have enough information now, or do you need more information?

I did a little digging into this issue because performance on lower end machines interests me personally. I plan on getting the numbers when I can figure out how, for now I did my own testing. I actually prefer to use a value of 'vm.swappiness=100' on lower memory machines. My reasoning is that the system is going to go into swapping regardless on say 384mb of ram (the lowest I get a usable desktop on the default install). So getting less used memory already into swap can improve performance for when it is needed for a more active application. As for testing, the goal is to get Firefox up as fast as possible.

With vm.swappiness=10, the system boots and is able to log in, though attempting to start Firefox brings the system to a halt as it swaps pages back and forth. Attempting to multi-task only slows things down more.

With vm.swappiness=100, after login the system has some disk I/O while idle at the desktop, but responsiveness is not hurt much. Starting Firefox actually brings it up in around 10 seconds, and I am also able to bring up the Gnome System Monitor to view process information as well.

To reproduce these results for yourself (even on a powerful machine) you can simulate out-of-memory pressure simply by passing a temporary (reverts to default after reboot) boot parameter to the system. To test:

1. Hold shift while booting, wait until grub loads. Highlight the designated kernel to boot, and press E. Go down to the line that starts with "Linux" and press "END"
2. Append a space, followed by: mem=384m
3. (NOTE) This command will tell the kernel to ignore all but that amount of RAM. You can change the above value to anything you wish, such as mem=256m
4. Boot into the system and test your current swappiness by trying to load Firefox. The benchmark is only meaningful if the machine is under memory pressure, so lower the value of the boot parameter if the system does not seem sluggish.
5. Try another value (such as vm.swappiness=100), reboot and start again at step 1.

As I said, I plan on getting the numbers, and am currently working out a way to do so. I should make my position clear I desire no change unless performance gains are drastic. The default value of 60 is sufficient as far as I see.

It seems to me that setting the value to vm.swappiness=100 has few downsides on a system with quite sufficient ram as well. Using the option on a machine with 3gb does not make it use swap at all.

@JFo, Scott

Could we please have another look at this issue? It seems everyone agrees that the default swapiness is inappropriate for desktop use yet somehow another release has slipped by without making the change. What in particular is needed to get things moving forward again?

- Ben

@Scott

I'm not certain I agree with your assertion that "this is a kernel issue and should be fixed in the kernel code." There is no chance this change will be taken upstream and there is a perfectly good usermode mechanism for overriding this value (sysctl(8)). This is in fact exactly why this value is exposed to user space. Given this, it seems strange to carry such an unnecessary patch for eternity in our kernel tree.

Perhaps the right solution would be to drop a file in /etc/sysctl.d from linux-image-generic?

- Ben

Here is another approach. This patch adds a Kconfig option for the default swappiness so this can be easily set in the per-flavour kernel configurations. I'll send this upstream as an RFC to see if this might be a workable solution.

The patch made it upstream for 2.6.37 so we now have the necessary knob.

Kernel folks, can we move forward on getting the default swappiness changed in the desktop kernel?

@Ben Gamari -- I cannot see that patch in v2.6.37? From the v2.6.37 source it seems unchanged, I also cannot see it in v2.6.38-rc1:

    int vm_swappiness = 60;

Also it is not at all clear that there is agreement on what is a good value for desktop, for example in comment #3 Robert says "swappiness=10 is a good setting for desktop" and in comment #33 Virgil states a contrary view "I actually prefer to use a value of 'vm.swappiness=100' on lower memory machines" (Virgil seems to have done a range of comparitive tests). Without sane guidance as to what is and is not the right value we are not in a position to change it even if we did have a knob.

Personally I have found desktop lumpyness has been most affected by the sched autogroup changes than anything for swappiness.

@Andy Whitcroft: I don't agree with your conclusion.

I've applied a swappiness reduction (from 60 to 10, and sometimes even from 60 to 1) on a whole range of various computers. Some 30 or 40 computers in total.

On *all* of those machines I saw a noticeable performance increase. On some machines, with 512 MB RAM or less, the performance increase was even spectacular.

It appears form my own, rather extensive, practical experience, that a swappiness of 10 is a good default for desktops. On 512 MB RAM or less, a swappiness of 1 is even better. This is confirmed by numerous reports on the Dutch Ubuntu forum, of which I'm an active member.

This depends a lot on the io devices. Consider a setup with a SSD for swap and system, and rotating disk for data. You
can then afford to swap much more to free memory for caching data from the magnetic media, compared to having
swap on magnetic media.

This bug was filed against a series that is no longer supported and so is being marked as Won't Fix. If this issue still exists in a supported series, please file a new bug.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

This shouldn't have been closed by an automatic script.

This problem is still present in all current Ubuntu versions, up to and including 11.10 Oneiric Ocelot.

Ben Gamari's patch has not landed upstream. The thread from Ben Gamari's attempt to submit this upstream is available here:
https://lkml.org/lkml/2010/11/1/500

Considering the reception my patch encountered, I'd say we won't be getting a CONFIG_SWAPPINESS upstream. Either a dummy package with a sysctl setting (thrown in /etc/sysctl.d) or an init script would work if we still think we need this knob.

Personally I would say this swappiness should still be resolved in some fashion. Since upgrading to oneiric on a few boxes that have 2GB of RAM i'm finding that compiz will capitalize a sizable chunk of memory which ends up using up a lot of swap after running for a day or two. The performance is abysmal at this point. There might be a problem causing compiz to use up this much memory in the first place, but the same thing happens if you leave thunderbird running with several mail accounts with multi GB mbox files.

So be it a ubuntu specific kernel patch to set a different default, a sysctl conf file, or an init script, I think this should still be fixed at least for the desktop scenario.

Still a serious problem in Precise.

i've read some of the comments but still don't understand why such an easy patch cannot be held easily by the ubuntu team.

there's a lot of area where ubuntu influence the way the kernel will work , with modprobe configuration files for example.

and /etc/sysctl.conf is already used to override some of the kernel settings.

Why not this one ?

It seems to me that's it's another bad sign that the ubuntu desktop don't really care about Pre-2006 computers with less ram and slower disks. it's where Lubuntu takes off.

Been using ubuntu for 5 years on the desktop, only just found out about this it sucks the ubuntu desktop pacakge should set this value