updatedb.mlocate uses 100% of the hdd IO. The OS is completely unusable.

Bug #1190696 reported by Curtis Lee Bolin
318
This bug affects 70 people
Affects Status Importance Assigned to Milestone
mlocate (Ubuntu)
Confirmed
Undecided
Unassigned

Bug Description

updatedb.mlocate uses 100% of the hdd IO. The OS is completely unusable. This can go on for an hour. I have been fighting with this on 3 separate computers for at least a month.

A very old related bug:
https://bugs.launchpad.net/ubuntu/+source/findutils/+bug/332790
That bug was marked as fix released for Hardy.

ProblemType: Bug
DistroRelease: Ubuntu 13.04
Package: mlocate 0.25-0ubuntu2
ProcVersionSignature: Ubuntu 3.8.0-23.34-generic 3.8.11
Uname: Linux 3.8.0-23-generic x86_64
ApportVersion: 2.9.2-0ubuntu8.1
Architecture: amd64
Date: Thu Jun 13 08:11:38 2013
InstallationDate: Installed on 2013-05-09 (34 days ago)
InstallationMedia: Ubuntu-GNOME 13.04 "Raring Ringtail" - Release amd64 (20130424)
MarkForUpload: True
ProcEnviron:
 TERM=screen
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: mlocate
UpgradeStatus: No upgrade log present (probably fresh install)

Revision history for this message
Curtis Lee Bolin (curtisleebolin) wrote :
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in mlocate (Ubuntu):
status: New → Confirmed
Revision history for this message
Morten Furbo (mfurbo) wrote :

As far as I can see the problem on my systems is that ionice is not working as expected on LVM volumes. Perhaps another way of deprioritizing updatedb.mlocate should be used.

Revision history for this message
Curtis Lee Bolin (curtisleebolin) wrote :

As of the last weekend, I updated all of my computers to pre-released Saucy (13.10); not once have I had this problem. Sorry i'm no help at pointing out what has changed from 13.04.

Revision history for this message
Anders Glent Buch (andersgb1) wrote :

Using Ubuntu 13.04 here.

Imagine being dependent on using Ubuntu for work during the day, and all of the sudden you need to just stop all activity for 1 hour (I do like breaks, but only when I choose them myself)! Since I'm no Linux expert, it took me quite some time to find out that it was a 'find' process eating up all my HDD.

This bug is exceptionally severe for me, and likewise for many others, I would assume...

Revision history for this message
Nec (nicolas-ecarnot) wrote :

Hi,

I found out that playing with ionice nor nice did not improve anything in that case. (that also was VERY severe to me)

The issue came from the fact that my computer mounts some remote filesystems, and find and locate went exploring those distant folders.

I solved the problem by changing /etc/cron.daily/locate and adding the "xdev" options to the find parameters.

# Global options for invocations of find(1)
FINDOPTIONS='-ignore_readdir_race -xdev'

Now, updates are still relevent, but cause no CPU harm nor wasted time :)

Revision history for this message
Wotim (t-yellow-eyes) wrote :

Same here with ubuntu 13.10. I searched what blocks every day my computer for more than 1 hour. First I found with iotop that something accesses my mount.ntfs where a 200 GB partition of the system hd is mounted. Then with ps I came to the find command of updatedb described here. This is a really annoying bug. I don't think that an inexperienced user can find out what blocks his computer for more than one hour. I am wondering that this bug exists with no change from ubuntu 13.04 until now.

Revision history for this message
Martin Borho (martin-borho) wrote :

Yes, this really annoying. This process seems not very "NICE"...

Revision history for this message
Edle8 (edle8) wrote :

Ubuntu GNOME 13.10. Confirmed the issue.

Revision history for this message
NoOp (glgxg) wrote :

 3.11.0-19-generic #33~precise1-Ubuntu SMP Wed Mar 12 21:16:27 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
linux-generic-lts-saucy

$ apt-cache policy mlocate
mlocate:
  Installed: 0.23.1-1ubuntu2
  Candidate: 0.23.1-1ubuntu2
  Version table:
 *** 0.23.1-1ubuntu2 0
        500 http://archive.ubuntu.com/ubuntu/ precise/main amd64 Packages
        100 /var/lib/dpkg/status

98% disk usage during updatedb.mlocate rendering the system useless until the process is killed or finished.
From atop:
DSK 98%
RDDDK 10100K
WRDSK 1052K

Revision history for this message
Hawk (beehock) wrote :

machine always slow to a halt when updatedb.mlocate is running.

Revision history for this message
aloctavodia (aloctavodia) wrote :

Still and issue on Ubuntu 14. 04

Revision history for this message
Ari Hauksson (imrucis) wrote :

It virtually makes the computer unusable while updating is going on. The bug should have a high priority. Strange that it has been going on for more than a year!

Revision history for this message
Guy (guy-maskall) wrote :

Yes, mlocate is a nightmare pain in the ... I get to work in the morning and my computer is so unresponsive with updatedb.mlocate eating up nearly 100% IO as shown on iotop. On my previous install, I had changed the service to run in the small hours of the night, but I might try this xdev flag as well.

A service like this shouldn't turn a machine into a 486 trying to run Windows 7!

Revision history for this message
Guy (guy-maskall) wrote :

I just realised I posted the comment below on the old bug threat, so posting it here as well to be more contemporary!

ps auxw shows me the ionice'd process:
root 6466 0.0 0.0 4328 356 ? S 07:35 0:00 flock --nonblock /run/mlocate.daily.lock /usr/bin/ionice -c3 /usr/bin/updatedb.mlocate

but the problem process, blocking IO is:
root 6467 1.0 0.0 5688 2220 ? D 07:35 1:25 /usr/bin/updatedb.mlocate
It's this that eats nigh on 100% of IO. Is it a subprocess that's spawned and isn't ionice'd?

Revision history for this message
jazzmale (jazzmale) wrote :

May be it is the same issue so still affects Ubuntu 14.04.1.
updatedb.mlocate uses 100% of the hdd IO. Not for an hour but 5-15 minutes.
At the same time wasting all memory and active swapping.

Revision history for this message
Nico (nnico) wrote :

I have this problem too and I have a question about Necs comment with reads:

"
Hi,

I found out that playing with ionice nor nice did not improve anything in that case. (that also was VERY severe to me)

The issue came from the fact that my computer mounts some remote filesystems, and find and locate went exploring those distant folders.

I solved the problem by changing /etc/cron.daily/locate and adding the "xdev" options to the find parameters.

# Global options for invocations of find(1)
FINDOPTIONS='-ignore_readdir_race -xdev'

Now, updates are still relevent, but cause no CPU harm nor wasted time :)
"

First I dont have a /etc/cron.daily/locate on 14.10 and I have only a /mlocate and there is no find command in there so I guess I can't cry to fix it this way? Or should I just and the FINDOPTIONS line in there?

I guess I will completely disable the cronjob for now.

Revision history for this message
dinamic (dinamic6661) wrote :

same on Ubuntu 15.10 :/

Revision history for this message
Sergio Benjamim (sergio-br2) wrote :

Same in 14.04.2.

Weird is that for some reason, it's downloading or 'rebuilding'* all my received/inbox emails from one account every time (yahoo.com.br, 2 or 3 times a day). *I can see the numbers growing up until about the total, ~4700

31.6.0

Revision history for this message
THPN (thpn) wrote :
Revision history for this message
philipp kretzschmar (kopernikus) wrote :

I am experiencing this one one 14.04 using a fully encrypted LVM file system. updatedb.mlocate hogs up nearly all of the CPU power making the system feel very sluggish.

Revision history for this message
h1repp (heinz-repp) wrote :

same here since a long time, and now still on 14.04.3, but it seems pretty useless posting here, not a single reaction from the developers since two and a half years ...

Revision history for this message
Valentin Lab (vaab) wrote :
Download full text (4.7 KiB)

I had the same issue, and found some information that might be of some interest.

If you have ``ionice`` on your system, it'll use it. But as a lot of people noticed, this doesn't change anything, which means that ``ionice`` is not working, which is a second issue.

Digging into this last issue, it seems that ``ionice -c3`` only works if the kernel scheduler of that device is set to ``cfq``.

You can check the current value with:

$ cat /sys/block/sda/queue/scheduler ## of course replace "sda" by your device's id.

(More info: http://serverfault.com/questions/485549/ionice-idle-is-ignored )

And you can change it runtime with:

$ echo "cfq" > /sys/block/sda/queue/scheduler ## you need to be root for that

(More info: http://stackoverflow.com/questions/1009577/selecting-a-linux-i-o-scheduler )

My system remains laggy actually even with this, but it seems better. Any feedback is welcome.

An important factor seems also the swap usage, if it is on the same device. For some reason, even
if I have plenty of free ram, my swap usage seems to rocket up when launching ``updatedb.mlocate`` (Goes from 0 to 2.5G of swap even if my RSS is only 4.9G on the 8G available.)
To check if it's your ``swap`` competing with ``updatedb.mlocate``, check you have enough free ram
to receive all your current swap and turn of swap:

$ swapoff -a

With this last option, my system is almost totally lag free when running ``mlocate.updatedb``. To be noted:
my RSS is not going up, so the swapping occuring usually is a decision of the kernel that here happens
to be a bad decision. Even if this solves my issue, this is not a viable solution as turning of swap is not
 benign. You might want to fiddle with the ``swappiness`` of the kernel to hint it not to swap as much
(More info: http://askubuntu.com/questions/103915/how-do-i-configure-swappiness ), but even at
swappiness=0, this doesn't change the behavior of the kernel with regards to ``mlocate.updatedb``.

$ swapoff -a ; sysctl vm.swappiness=0 ; swapon -a ## the command I used to test

By looking at memory consumption, ``updatedb.mlocate`` is using very few of it, and remains stable.

Could it be related to the filesystem ? My filesystem is ext4 for the system part, but the biggest part is
on Btrfs. I would safely bet that this is what is happening: the filesystem driver is looking everywhere
and asks for more buffers to store all these paths, but this is not the right decision here. (this is
what makes for instance a first "find /" being noticeably sped up the second time you are launching it,
but here we surely won't benefit of this because ``updatedb.mlocate`` is run once a day.).

Finaly, a good way to help updatedb do its jobs quickly is probably to limit the folders it'll scan. Looking at
the output of:

$ updatedb.mlocate -v

or you could consult your current database:

$ locate /

And even get some advanced reports as with:

$ locate / | cut -f 1-4 -d / | uniq -c | sort -n

and:

$ locate / | wc -l # gives you the number of file stored in updatedb database

These commands will give you some idea of where it spends some time and you might spot a lot of places that are huge and that
can't be really usefull...

Read more...

Revision history for this message
Alexander Eifler (eifx) wrote :

Very usefull comment on askubuntu:

https://askubuntu.com/questions/268130/can-i-disable-updatedb-mlocate

"Edit the file /etc/updatedb.conf and set PRUNEPATHS with the directories you DO NOT want to search."

I disabled the /mnt for example because there is a mount-point to a ntfs-device. So, I have now less I/O interactions and, because of the ntfs-driver, less CPU interactions.

My notice: Have a look to PRUNEPATHS and disable mount-points or large directories with no interesting content (maybe large git or svn dirs or so)

Revision history for this message
R. Diez (rdiezmail-ubuntu) wrote :

I have recently hit this issue again. My system crawled to a halt and I thought for a moment that I was running Windows 10 and its Windows Update. 8-)

I have written a "conflicting" deb package for mlocate in order to prevent it from being automatically installed again because of some suggested or recommended dependency. You may find it useful. It is here:

https://github.com/rdiez/Tools/tree/master/MlocateConflictingPackage

Revision history for this message
quchunguang (quchunguang) wrote :

I have this issue either.

Ubuntu 17.10 64bit

Revision history for this message
Nishant Elkunchwar (thecountoftuscany) wrote :

Facing this issue on Ubuntu 16.04 and looks like the best solution right now is to just uninstall the mlocate package and install the MlocateConflictingPackage (https://github.com/rdiez/Tools/tree/master/MlocateConflictingPackage) to avoid installation of mlocate again due to some updates.

Revision history for this message
Jacopo (jacobest-x) wrote :

Hi, I just faced this issue on Ubuntu 18.04, at least I was able to Google and be able to find this issue and Nishant comment...

Revision history for this message
Jason Grammenos (jason.grammenos.agility) wrote :

I have this issue on 16.04 (recently in place upgraded from 14.04), it makes the physical machine basically useless. And this issue persists for hours. Today is the second time it has happened on the same machine. last time I kill the process after some number of hours of waiting.

Revision history for this message
Mika Huurre (mika-huurre) wrote :

Ubuntu 18.04.1 LTS, same issue.

Revision history for this message
Mika Huurre (mika-huurre) wrote :

killall updatedb.mlocate and reboot. Seems that problem solved (for now).

Revision history for this message
i30817 (i30817) wrote :

Crashes my (terrible, overheating computer) for something do not even want. Should really be a option to get rid of alll the 'indexer needed' features.

Revision history for this message
Jarno Suni (jarnos) wrote :

mlocate does not seem to be included in Ubuntu 20.04 by default, but is included in Xubuntu 20.04.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.