hud-service frequently tries to accesses non existing files causing high disk/network activity

Bug #998759 reported by Manfred Thole
82
This bug affects 17 people
Affects Status Importance Assigned to Milestone
Application Menu Indicator
Confirmed
Low
Unassigned
System Load Indicator
Confirmed
Low
Unassigned
indicator-appmenu (Ubuntu)
Confirmed
Low
Unassigned

Bug Description

/usr/lib/indicator-appmenu/hud-service is frequently calling access() for the following three files:

access("<home>/.cache/indicator-appmenu/hud-usage-log.sqlite-journal", F_OK) = -1 ENOENT (No such file or directory)
access("<home>/.cache/indicator-appmenu/hud-usage-log.sqlite-wal", F_OK) = -1 ENOENT (No such file or directory)
access("/usr/share/indicator-appmenu/hud/app-info/multiload.hud-app-info", F_OK) = -1 ENOENT (No such file or directory)

which apparently do not exist (excerpt from strace log, <home> is users home directory).

For NFS mounted homes this causes a constant network traffic of ~300kB/s down and ~50kB/s up in idle state.
On slower machines this causes a more than noticable CPU load (with and without NFS mounted homes).

ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: indicator-appmenu 0.3.97-0ubuntu1
ProcVersionSignature: Ubuntu 3.2.0-24.37-generic 3.2.14
Uname: Linux 3.2.0-24-generic x86_64
NonfreeKernelModules: fglrx
ApportVersion: 2.0.1-0ubuntu7
Architecture: amd64
Date: Sun May 13 15:36:15 2012
ExecutablePath: /usr/lib/indicator-appmenu/hud-service
InstallationMedia: Ubuntu 11.10 "Oneiric Ocelot" - Release amd64 (20111012)
SourcePackage: indicator-appmenu
UpgradeStatus: Upgraded to precise on 2012-04-28 (15 days ago)

Revision history for this message
Manfred Thole (tanfred) wrote :
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in indicator-appmenu (Ubuntu):
status: New → Confirmed
Revision history for this message
KVV (kouzminv-yahoo) wrote :

If indicator-multiload is running, each indicator update cycle causes this hud-service activity. With the default 500ms indicator refresh time, my workstation is constantly sending ~300kB/s and receiving ~670kB/s from the NFS server hosting home directory, as well as ~9% CPU usage (mobile sandybridge CPU).

Revision history for this message
Sebastien Bacher (seb128) wrote :

Thank you for your bug report, it's not likely that those stats create the performance issues though, it's rather an indicator-multiload issue, sending status update over dbus every 500ms

Changed in indicator-appmenu (Ubuntu):
importance: Undecided → Low
Revision history for this message
Bilal Akhtar (bilalakhtar) wrote :

I think hud-service should (cache?) store stuff in memory in such situations, if calls are being made repeatedly, for example. But yeah, it's a low-priority task on this end, the rest of the load lies on indicator-multiload.

Changed in indicator-appmenu:
status: New → Confirmed
importance: Undecided → Low
Revision history for this message
KVV (kouzminv-yahoo) wrote :

I see two problems:

1) indicator-multiload is causing (via dbus?) hud-service to do "something" (update/reload cache?) each time indicators get refreshed. indicator-multiload does not do any "strange" network activity by itself, strace shows quite sane behavior (access to /proc and /sys and such).

2) each hud-service "update" initiated by indicator-multiload is casuing some 0.5MB network exchange (if home directory is on NFS), about half with nfs port and half with nfs lockd. I tcpdumped both sessions. Lock protocol is all binary, looks like repeating lock requests. On nfs port, it does the same over and over again. I counted this snippet called 88 times during one hud-servide "update":

q.........O.W.7|N.O... ..yO... ..y............SQLite format 3......@ .........................................................................-.!
.....I....I.........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................W.../..windexapplication_indexusage.CREATE INDEX application_index on usage (application)\........tableusageusage.CREATE TABLE usage (application text, entry text, timestamp datetime)

As a workaround, I can:
1) reduce frequency of indicator refresh
2) kill hud-service (and so this network activity stops)

Changed in indicator-appmenu:
importance: Low → Medium
Changed in indicator-appmenu (Ubuntu):
importance: Low → High
Changed in indicator-appmenu:
importance: Medium → High
tags: added: performance
summary: - hud-service frequently tries to accesses non existing files
+ hud-service frequently tries to accesses non existing files causing high
+ disk/network activity
Revision history for this message
KVV (kouzminv-yahoo) wrote :

Technically this only affects the user if indicator-multiload is installed (non-default config but it's in the standard repositories in 12.04) AND home directory is on NFS (uncommon). If home directory is on local drive, this will probably have little impact due to caching.

But indicator refresh should cuase zero traffic with the NFS server, otherwise a network of tens or hurdreds NFS workstations will cause quite a troble to the server and the network.

Revision history for this message
Ted Gould (ted) wrote :

An indicator refresh would cause a check of the usage database if there was a search active, as it could possibly change the ordering of the search. Though, it sounds like that's not the case here.

Not sure of the exact cause, but a work around would be to tell the HUD to not save usage data so it'll only use an in-memory database:

  $ gsettings set com.canonical.indicator.appmenu.hud store-usage-data false

Which would mean that the locking files wouldn't be checked as the database is read.

Revision history for this message
KVV (kouzminv-yahoo) wrote :

> $ gsettings set com.canonical.indicator.appmenu.hud store-usage-data false

This worked, traffic dropped to zero!

Revision history for this message
gcc (chris+ubuntu-qwirx) wrote :

HUD is repeatedly opening and closing the database. That is very inefficient on any system, but especially over NFS. (I don't see 2 DBus requests per second as inappropriate, unlike comment #4).

Can't it just keep the database open? Nobody else should be using it at the same time.

Michael Hofmann (mh21)
Changed in indicator-multiload:
status: New → Confirmed
importance: Undecided → Low
Ted Gould (ted)
Changed in indicator-appmenu:
importance: High → Low
Changed in indicator-appmenu (Ubuntu):
importance: High → Low
Revision history for this message
Jean Christophe André (progfou) wrote :

I confirm on my side that the gsetting suggestion from Ted Gould (comment #8) is working for a user with his homedir over NFS. I have now put it in my default Ubuntu deployment strategy.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.