For large workspace bazaar commands can only be executed after a couple of minutes.

Will give recommendations when I get a chance to look at code.

This is probably caused by the resource change listener as we wait for all bzr move commands to finish before showing qcommit dialog for example. Do you see a message "Waiting for Bazaar to move files" in the log view?

No, but its even worse. If I shutdown eclipse and these ignore tasks are still getting executed in the background.

To reproduce:
Import many eclipse projects (mine happen to be in the same branch. I opened up 27 projects with +- 10500 files in them, in total.)
This is alot but not all the projects I open up normally.

Immediately after you import these projects minimize the progress bar and run qdiff.

You will notice that the diff operation doesn't get to take place. Also if you run "ps aux | grep bzr" in the terminal you will see is-ignored commands getting executed etc. Like I have mentioned this will continue even after eclipse is shutdown.

BTW: To get them to stop, the thread that processes the queue must be a daemon thread.

Ok I've added the change that stops the queue getting worked down even after eclipse has shutdown. But thats the easy part to this problem.

The problem here is that hundreds of threads are created (over time) to check if files that entered the workspace are ignored by bazaar. So we will need to discuss a few solutions to avoid this:

One that comes to mind is that we can run a "bzr ignored" command and then populate ignored-cache this way. Rather than doing it one for one per file. Also when the cache expires another ignore operation can take place to repopulate the cache etc.

I've added a new preferences option on trunk and now all commands
executed are logged when this option is on. You could use this to see
how many is-ignored commands are executed.

I don't think it's the creation of the threads to run the commands that
takes the time is it? We could use a thread pool if it is but I would be
surprised if that turned out to be the reason why it is slow.

Sorry I might have worded by comment incorrectly.
All I know is that when I open these projects for the first time it takes a couple of minutes before I can execute a qbzr bazaar command. Also I can see that it executes bzr is-ignored for each file and directory (well atleast is looks like it) in each project one for one. (in my case, probably 10500 times)

So I suspect that the delay (before my qbzr command gets executed) comes from the fact that these is-ignored commands are still being "worked down".

I've looked at the log files and its clear that there is an is-ignored command for every file in my project. I think a once off "bzr ignored" to prepopulate the cache might make a lot of sense.

That's strange - I don't see that on my machine. Can you set a breakpoint in the BazaarUtilities.isIgnored method and see where it gets called from?

A possible improvement would be to check the time stamp of the .bzrignore file and only expire the cache if we notice this has changed. At the moment the cache expires every 10 seconds. But it definitely looks like a bug that you get so many is-ignored commands....

>
> That's strange - I don't see that on my machine. Can you set a
> breakpoint in the BazaarUtilities.isIgnored method and see where it gets
> called from?
>

It gets called from:
removeUninterestingChanges()

> A possible improvement would be to check the time stamp of the
> .bzrignore file and only expire the cache if we notice this has changed.
> At the moment the cache expires every 10 seconds. But it definitely
> looks like a bug that you get so many is-ignored commands....
>
>
That's a very good idea, for handling cache expiry policy. I still think
that pre-populating it with the bzr ignored command will
make it more efficient (single process instead of many processes etc)

Remember to reproduce, start with an empty workspace and then import a
project with many files (the project must be version controlled).

Ok I can reproduce it. I think you are right we could call "bzr ls --ignored" to populate the cache as well.

Another fix would be to ignore added files if the branch was added (eg by importing the projects).

Just as a reminder, make use of "bzr ls -R --ignore" and track changes made
to the .bzrignore file

Actually I don't think we can use the ls --ignored command. The reason is that we will also have to invalidate the cache whenever a new file is added as this file could potentially match an ignore pattern.

Maybe test if the newly added file is in the ignore cache.
If its not then you can test it via is-ignored command (since there won't be
too many newly added files by the user. ) and just insert it, without doing
a invalidation of entire cache.

On import of a project the files will already have been ignored and when the
come up as "added" events the will be found in the ignore cache and the
is-ignore check won't be done.

So the cache should only be invalidated when .bzrignore file changes. Unless
I'm missing something....

What happens if the user has a user defined ignore file? (ignore file not in
branch) That might also have to be tracked.

I've changed it so that it doesn't maintain a cache at all but only for the current set of changes that are being processed. Can you try with the latest trunk and see if the problem is fixed for you? I can still produce sometimes when the "Add files automatically" option is on but I'm working on that....