Hot Bugs algorithm works poorly for some projects, making it a bad default listing

Would it be possible to specify the default search type on a per project
basis?

On 04/07/2010 12:06 PM, Nikodemus Siivola wrote:
> Public bug reported:
>
> For eg. SBCL the Hot Bugs listing is fairly pointless: the heat
> algorithm doesn't produce appreciable differences for our bugs, and
> lists uninteresting bugs prominently.
>
> Recently Touched would be a better metric of "worth putting in the
> shortlist" for us, as would a sorting based in Status/Importance.
>
> See:
>
> https://bugs.launchpad.net/sbcl
>

I wondered if the hot bugs list was useful for projects with low heat values. This confirms it's not. I also want more than just a hot bugs list myself. I get tired of looking at the same bugs. Though I do see some variation over time, the list changes slowly.

Rather than removing the hot bugs list or making it configurable, how would you feel about multiple lists on the bugs home page, similar to the overview page's latest bugs, questions, etc. lists -- i.e. a hot bugs list plus a recently changed bugs list and a recently reported bugs list? I think the shortened form of displaying bug lists in two columns could be used to fit more on the page in a smaller space, too.

How does this sound?

Multiple lists sounds fine.

(Apropos, would it be possible to nudge heat based on eg. tags like "hot" or "cold"?)

On 7 April 2010 20:22, Nikodemus Siivola <email address hidden> wrote:
> Multiple lists sounds fine.
>
> (Apropos, would it be possible to nudge heat based on eg. tags like
> "hot" or "cold"?)

Not at present. There's been some discussion about how we could extend
heat to take account of tags and suchlike on a per-project basis, but
we've come to no conclusions and I'm pretty sure when I say we aren't
likely to look at heat beyond bug fixes for the next few months at
least.

I'm sceptical about having a second list on the bugs home page, but willing to be convinced. I really really don't want it looking like the project overview page.

On 8 April 2010 04:43, Deryck Hodge <email address hidden> wrote:
> I wondered if the hot bugs list was useful for projects with low heat
> values.  This confirms it's not.  I also want more than just a hot bugs
> list myself.  I get tired of looking at the same bugs.  Though I do see
> some variation over time, the list changes slowly.
>
> Rather than removing the hot bugs list or making it configurable, how
> would you feel about multiple lists on the bugs home page, similar to
> the overview page's latest bugs, questions, etc. lists -- i.e. a hot
> bugs list plus a recently changed bugs list and a recently reported bugs
> list?  I think the shortened form of displaying bug lists in two columns
> could be used to fit more on the page in a smaller space, too.

My highschool teacher used to call this a 'shotgun answer': writing
down everything you can think of in the hope some of it hits. :) I
think one list is much cleaner and makes it easier to actually scan;
presumably the point of this is to give an overview of activity in the
project.

To me the most interesting default is 'most recently changed' but
that's just a personal feeling.

How about if we run a little script across Launchpad's access logs to
find out which sort mechanism is the most popular when no search terms
are specified. Then consider making that the default. At any rate it
would be interesting to see the results.

--
Martin <http://launchpad.net/~mbp/>

Fair point about the shotgun answer approach. :-) I taught English once, despite my accent, so you win me over completely with this argument. :-)

I agree, too, that some data about what people actually do would be nice to have. I'll try to get this together before the week is out.

Another option comes to mind, and I worked up this quick mockup. The mockup is based off an old one of intellectronica's so that I could work more quickly, so some parts of the "framing" no longer apply. But the point is this -- a main list of recently touched or importance bugs, depending on what the data says people use more, and then add in a small "hot bugs" list in compact form to call attention to problem bugs. We could still have the link to all hot bugs, which the mockup doesn't show.

If people are warm to the general idea, then we can do a proper mockup in the current page style which doesn't crowd things so much. Or perhaps this leads us to an even better idea.

On 15 April 2010 05:35, Deryck Hodge <email address hidden> wrote:
> Fair point about the shotgun answer approach. :-)  I taught English
> once, despite my accent, so you win me over completely with this
> argument. :-)

:-)

> If people are warm to the general idea, then we can do a proper mockup
> in the current page style which doesn't crowd things so much.  Or
> perhaps this leads us to an even better idea.
>
> ** Attachment added: "Bugs home page with heat and activity both displayed"
>   http://launchpadlibrarian.net/44185431/bugs-index-heat-and-activity.png

I think what I would like is actually to see the bugs ordered by
most-recently-touched, but showing their flames, then you perhaps get
the best of both worlds. The precise date when something was changed
is not actually very interesting, it's enough to just see the last few
days (or whatever). And it will still keep heat a bit visible.

My sense is that the heat algorithm is not yet tuned enough that it
really highlights the bugs you must always be aware of.

I think that mockup still tends to make the page a bit cluttered; I
would rather eventually see that space used for eg putting a few of
the most popular advanced search items onto the page, like a checkbox
for "include closed bugs".

--
Martin <http://launchpad.net/~mbp/>

I think <http://pastebin.ubuntu.com/414679/> (untested) should tell us
which ones are most popular if it's run across the production server
logs.

(Or more precisely, the same script adapted to count and summarize,
once I know it basically works.)

--
Martin <http://launchpad.net/~mbp/>

spm was kind enough to run this query over some recent data and the results are interesting:

  23015 -importance
   1104 -datecreated
   1083 -date_last_updated
    160 status
    125 datecreated
     42 -heat
     28 date_last_updated
     26 milestone
     25 -users_affected_count
     10 -number_of_duplicates
      6 targetname
      6 -message_count
      1 title

I suspect importance is overweighted because it's the default, and will be hit by the default advanced search, by people just pressing search to get a fuller list, and by robots. How much it is overweighted is hard to tell, but if we changed the default that would give us some data.

What this does tell me is that all of importance, datecreated or date_last_updated are much more popular with users than heat is.

Deryck and I have discussed this and I think we've narrowed the solution space a bit:

1. No shotgun, we're keen on having a single list of bugs in that place. Whatever improvement we choose it has to be an improvement to this list.
2. Heat is not bad already for many projects, and can only improve. It would be a shame to abandon it for something you can easily find using the filters or advanced search, even if it's proving popular (like -importance -datecreated and -date_last_updated).

Deryck suggested finding a way to bring more recently active bugs into the list. One way to do that might be by tweaking the heat formula itself. A different option would be to leave heat as is for now, and make the list displayed on the bugs homepage display recently active bugs, but taking heat to into consideration).

istr some past discussion that heat would fade over time (indeed the
name almost implies it), so that could be a good synthesis here. If
you get not 10 points per subscriber, but 10 points when a subscriber
is added or x points when a comment is made then it may show more of
"what's hot now."

--
Martin <http://launchpad.net/~mbp/>

Tom and I have worked on ideas for bug heat calculation for the last few days and talked about it among bugs team developers, and we've got plans together now to add points for activity and also more rapidly fade heat on inactive bugs. The hope is that these changes will make currently hot bugs even hotter and inactive bugs colder and that this change will be enough to cause the hot bugs list to be useful on more projects.

We're trying to get this landed over the next day or two, so that the change has as much time as possible on edge and staging before the 10.04 rollout to see if the changes do actually prove useful.

We believe this bug can be considered fixed once the following bugs are fixed:

Bug 567375
Bug 567379
Bug 567439

IMHO, the reason datecreated and date_last_updated are popular search sorts is because as a developer you only care about bugs in the current development version.

Since bugs reported against the current development version by definition will be amongst the newer bugs, sorting by datecreated will make these be in the top (mixed with other uninteresting bugs that just happened to be reported recently).

A better solution would be to make Launchpad track the versions of the package that were tested, and to allow filtering to only see bugs of that version. The default listing should show only bugs relevant for the version the user has, much like how PPAs only show packages for the Ubuntu version that the user is on.

A benefit of this is that it makes the bug lists look much less overwhelming. :-)

I've been experimenting with using scripts to tag X bugs with the distro version they affect, so I can ignore everything except lucid bugs. It's worked *amazingly* well. I'd be happy to elaborate at length on my thoughts on how to use this methodology more widely in launchpad.

Separate thought regarding date_last_updated. I use that search a lot when I'm in bug triager mode to look for replies to questions I've asked. However it's crude because it does not distinguish between comments from bug reporters vs. comments from triagers (or even from myself).

What I think is really wanted is sort of an "inbox" listing bugs with new messages, sort of analogous to how some forums work.

On 24 April 2010 14:49, Bryce Harrington <email address hidden> wrote:

> What I think is really wanted is sort of an "inbox" listing bugs with
> new messages, sort of analogous to how some forums work.

This would be a fascinating concept to add in across Launchpad.
Imagine if you could look at your review queue and see those that have
new messages in bold, or see a timeline view. But of course it's much
larger than this bug.

--
Martin <http://launchpad.net/~mbp/>

I'm marking this fixed since the bugs linked above are fixed.