tracker file system miner misses most files on Ubuntu 16.10

Bug #1644651 reported by Richard
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tracker (Ubuntu)
New
Undecided
Unassigned

Bug Description

This still works for Ubuntu 16.04 tracker 1.6.2-0ubuntu1.1

On Ubuntu 16.10 tracker mining of the file system stops mid-way, most folders are not re-cursed into.
- for a test system with 1 or 2 folders, it looks to be working

REPRO
Log in and start a terminal window
1. Configure
gsettings get org.freedesktop.Tracker.Miner.Files index-recursive-directories
['&DOCUMENTS']
- ensure that 300 or more folders are to be indexed
- to make changes launch the app Search and Indexing, tab Locations
2. Run
tracker reset --hard <<<y && tracker daemon --start && tracker daemon --list-processes && tracker daemon --follow
# possible problem: does not launch a 'tracker' process
wait for: File System Idle
3. Examine Indexed Results
tracker status --stat | grep Folder
  nfo:Folder = 3
find Documents -type d | wc --lines
559
BUG: those values should be same
4. List the directories that tracker failed to index
find Documents -type d -exec bash -c "if [ \$(tracker info \"{}\" | wc --lines) -le 4 ]; then echo \"{}\"; fi" \;
BUG: Lots of directories

Downgrading to 1.6.2-0ubuntu1 still does not work on Ubuntu 16.10, although that version works oj 16.04
- ie. tracker won't run at all under 16.10

GETAROUND
using kvm, create a virtual machine with 16.04, map your home folder there and run Desktop Search in your vm
The command for mounting a mapped folder in the guest: sudo mount -t 9p -o trans=virtio,version=9p2000.L /home/user /home/ubuntu/user

ProblemType: Bug
DistroRelease: Ubuntu 16.10
Package: tracker 1.10.0-1ubuntu1
ProcVersionSignature: Ubuntu 4.8.0-27.29-generic 4.8.1
Uname: Linux 4.8.0-27-generic x86_64
ApportVersion: 2.20.3-0ubuntu8
Architecture: amd64
CurrentDesktop: GNOME
Date: Thu Nov 24 13:28:36 2016
SourcePackage: tracker
UpgradeStatus: Upgraded to yakkety on 2016-11-16 (8 days ago)

Revision history for this message
Richard (ismail-a) wrote :
Revision history for this message
Richard (ismail-a) wrote :

by using dconf org.freedesktop.Tracker.Miner.Files verbosity 'debug'
and running it
in syslog tracker-miner-f finds all top level entries in Documents
None of those directories are stored or traversed, however
tracker-miner-f[11630]: Found 1 directories, ignored 0 directories

One can check:
grep --text tracker-miner-f /var/log/syslog | grep Processing |
less
- lists entries looked at
grep --text tracker-miner-f /var/log/syslog | grep Found | less
- lists everything found for those entries

broken broken broken

Revision history for this message
Richard (ismail-a) wrote :

As of 1/6/2017, this has somewhat improved
Tracker recurses but:
- without a reset only finds 90% of content
- with a reset, does not find everything anyway

1. Tracker still only finds 90% unless it is reset. So, reset like:
tracker reset --hard <<<y && tracker daemon --start && tracker daemon --list-processes && tracker daemon --follow

1b.Optionally, in a second window you can see file-by-file progress like:
tracker daemon --watch

2. Sometimes Extractor does not start: when File System reaches Idle, Extractor should start within 2 s. If it does not, go back to 1. If it fails again, you are unlucky.
- Extractor may take long time like hours to complete, but provides progress every 10 minutes.

3. Whatever you have now is the best you are going to get. There are some amount less than 10% of content that tracker is not going to find.

to see how many folders tracker found:
tracker status --stat | grep Folder
  nfo:Folder = 1646

dpkg --status tracker | egrep "(V|Pa)"Package: tracker
Version: 1.10.0-1ubuntu1

Revision history for this message
Will Cooke (willcooke) wrote :

Removing wayland tag since I don't think it's relevant here.

tags: removed: wayland-session
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.