Fix Committed/Released distinction is confusing and Fix Committed is not functionally different to Confirmed/Triaged/In Progress

This is currently a discussion in QA/Mobile QA about the need to confirmation of fixed issues and how there is currently no means of tracking the fact that a fixed issue has been confirmed by testing.

My initial reaction was to use "Fix Committed" as a catch all for development, so when development pushed the code to the release repo, the status would change to Fix Committed.

Once the fix is in this Committed state, QA would know that the code is available and could then download. If the fix was then verified to be fixed then the bug state would move to "Fix Released" so the greater community would know that they could now be assured that the fix was there.

The current fix verification system relies on tags which cannot be tracked or used as the basis for performance matrices. A move to this process would free Dev from having to worry about two status settings, and give QA a way of showing a bug fix was tested and proven.

Some projects have zero levels of fix verification, some have one, some have more than one. So using a bug status to track verification would result in projects having different meanings for statuses, which we want to avoid. Chris, what sort of tracking do you want for tags, and what do you mean by "performance matrices"? (Remember that counting the number of issues fixed/verified in a period is a terrible measure of performance, because issues can vary greatly in complexity.)

I propose to use target milestones and release series instead of (2) and (3): when a milestone becomes inactive, the bug can be considered "released" for the specified release series or the active ones.
Also, instead of showing "Fixed" every time (which can be ambigous), I think that it would be better to display "Fix committed" or "Fix released" automatically, but keeping a single item in the list of +ediststatus.

From a user perspective, the distinction should be made clearly in such a way that
we know when the bug will be fixed by a standards update proceedure.

The distinction should be clear in the following senses:

1 - The fix will be released in such a way that a standard update procedure will
patch it. The date of the release for standard updates would be a valuable
information.

2 - The fix is known but will NOT be released for all distributions. What to do
for each distribution should be stated, with something like:

  Patch cannot be applied / Patch can be manually applied (howto)

3 - Fix will only be possible in a future distribution.

I also feel that we would be better off without the Fix Released status. My main concern is not that it's entirely useless, but rather than it means completely different things for different projects. As such, these concepts would be better expressed using tags. The problem with tags is that currently hang off bugs, not bug tasks (the lines representing the manifestations of a bug in different pieces of software). For complex bugs that are manifested in many projects, it's very useful to know whether the fix was released in one of the projects, so if we want to get rid of this status we'll have to come up with a way to capture and display this information.

The current summary sounds lovely.

@intellectronica this seems orthogonal to tasks-in-projects.

2009/8/28 Robert Collins <email address hidden>:
> @intellectronica this seems orthogonal to tasks-in-projects.

I agree, and also agree mpt's description is very nice.

It seems to me that projects track this at present by just putting
freeform text into the closing comment on the bug.

If people want to distinguish between "fix exists" and "fix has been
merged to the release branch", which is the main thing we previously
used fix committed vs released for, then they can do that with merge
proposals.

> (1) "Fix Committed" and "Fix Released" be merged into "Fixed";

I think this can usefully be done independently of the others. They
would all be useful but I don't think fix committed vs released very
tightly relates to any of them - for example users deduping bugs are
as likely to have an old release as to have the current release and
hitting a fix-committed bug.

> (2) a text field be introduced for bugtasks, so developers (and Soyuz) can record the version/versions in which a bug was Fixed;

This is like structured infestation?

> (3) project maintainers be able to configure how long a bug report remains visible in default search results after the bug is Fixed (for Ubuntu this might be 12 months, while for Launchpad itself it could be 60 days);

That would rock, and I think even just starting with a global
arbitrary six-month "recently fixed" category would be a start and
would give some data on whether it's useful.

> (4) the "Is the bug you’re reporting one of these?" list treat bug statuses just like the normal search does, including fixed bugs only if they were fixed recently.

And see also <https://bugs.edge.launchpad.net/malone/+bug/277352>.
Perhaps eventually all searches should include recently-fixed bugs.
(Or perhaps even on a different timescale - bugs fixed last week are
likely still interesting, bugs fixed a year ago are generally not
relevant.)

--
Martin <http://launchpad.net/~mbp/>

I think we should do this. We are not getting a lot of value from the distinction, but we are getting a lot of problems.
Being able to link a branch and a merge proposal to a bug is in many ways already "fix committed". Granted we may need to present things a little differently and polish that experience and UI a bit, as well as other features that may come up (linking to branches outside of launchpad?).

I would sit down and think how we could present the basic case for this, and come up with a plan to get this done.
We need to be sure that we communicate this change as widely and clearly as possible.

I'm happy to hear you say that.

One part of that plan needs to be how existing fix committed bugs are to be handled. I suppose it is best to bump them back to in-progress, or perhaps to just block new bugs going into that state.

mp's give a much better and more nuanced view of how far the bug is from being started to finally fixed.

I'd be happy to help with the plan, since it's been churning away in my mind for a few years now. ;-) For example, I suggest this procedure for dealing with existing "Fix Committed" bugs:

1. Where a bug is "Fix Committed" for a project, change that status to "In Progress" and simultaneously add the tag "projectname.fixcommitted". Similarly for a package, change that status to "In Progress" and simultaneously add the tag "distroname.packagename.fixcommitted". Do not send mail notifications for any of these changes.

2. When finished, send one mail notification to the bug supervisor of each project/package that has any ".fixcommitted" bug reports, giving (1) the number of reports affected and (2) the link to a search results page listing them all (oldest first). That makes it as easy as possible to clean them up.

3. For as long as there is at least one ".fixcommitted" bug report for a project, package, or distribution (up to, maybe, six months later), show a canned link to that same search results page from the main Bugs page, as a reminder that the reports need updating.

Good plan mpt :)

One very nice aspect of the distinction between Fix Committed and Fix Released is that it makes the buildup of unreleased work quite obvious. Although we shouldn't necessarily block on actually doing it, we ought to have a plan on how to provide an equivalent in the absence of FC/FR.

2010/1/25 Jonathan Lange <email address hidden>:
> Good plan mpt :)

+1

> One very nice aspect of the distinction between Fix Committed and Fix
> Released is that it makes the buildup of unreleased work quite obvious.
> Although we shouldn't necessarily block on actually doing it, we ought
> to have a plan on how to provide an equivalent in the absence of FC/FR.

I have another bug asking that we provide a count of the In Progress
bugs and a link to them. This is a good thing for projects to monitor
anyhow. Doing this would make mpt's third step probably unnecessary.

--
Martin <http://launchpad.net/~mbp/>

I find the distinction very helpful actually.

Fix Committed = Code is in bzr repo attached to the bug. If you need the fix you can get it here.
Fix Released = Code is in one or more released versions of the project (as targetted).

I'd hate to see "Fixed" appear - or if it does some meaningful way of representing the above is then needed. Couldn't there be a middle ground where either projects can configure this themselves, or where you have all three options?

Alex

On Thu, Jan 28, 2010 at 09:19:04AM -0000, Alex Harrington wrote:
> I find the distinction very helpful actually.
>
> Fix Committed = Code is in bzr repo attached to the bug. If you need the fix you can get it here.
> Fix Released = Code is in one or more released versions of the project (as targetted).
>
> I'd hate to see "Fixed" appear - or if it does some meaningful way of
> representing the above is then needed.

The above is already represented more explicitly: when there is code in a
bzr repository related to the bug, it gets a branch link.

--
 - mdz

I don't understand how this bug relates to the recently-landed feature
of Launchpad automatically flipping bugs from committed to released.
Is that just meant to be an interim fix?

--
Martin <http://launchpad.net/~mbp/>

Not if the related branch isn't ready to be used yet. We'll often link a branch to a bug as it's being worked on - that doesn't however mean that the issue is resolved in that branch.

It's also harder to see that from the milestone overview page - which we use as a todo list for a release.

Alex

@Martin, yes, the other was unrelated to this, done by Curtis in his spare time for his own personal itch.

Martin.

My work was indendent of this bug, but my work implicitly solves some issues that are not being discussed in the bug. As a release manager of many projects I need to update bug statues to communicate that fixes are available for users (not engineers). I stopped doing this 9 months ago because I got complaints that I was stealing karma from the assigned engineer. So I would ask the engineer to close his bugs, but this is not an urgent matter for many.

This was a very frustration matter. I changed how karma was allocated and automated the status updates. Even though I have backout my change to update the bug status when a release is created, karma is still awarded to the bug assignee. Any automatted process that updates status from an inferred state will award karma to the correct user.

Sorry, I didn't mean to sound like I was complaining about the patch
you landed. I think it's great that you scratched your own itch and
if we were still using fixreleased in the manner you do I would have
hugely appreciated it. I was just wondering where Launchpad was
heading in the future.

--
Martin <http://launchpad.net/~mbp/>

I always assumed "Fix Committed" meant "bugfix merged to trunk" while "Fix Released" meant that if the user upgraded to the last release she would no longer see the bug. As a maintainer of a few open-source projects hosted on launchpad I'd be a bit sad to see the distinction disappear.

The Maemo project has a problem where users complain about bugs marked "RESOLVED: FIXED" by saying "no it's not, I upgraded to the latest version and the bug is still there!" with the developers going back saying "it's fixed in $VCS and will be released with the next version, I just don't want to see this bug among the list of open bugs as it distracts me". I usually chime in with a suggestion to implement Launchpad-style fix-committed/fix-released distinction when this issue comes up in Maemo discussions.

Point (2) from mpt's summary only seems really useful once LP has a notion of what those versions actually mean. Sure, it would be nice to record the data rather than systematically losing it, but I think adding support for that needs to be on somebody's plan somewhere. A date-based cut-off is distinctly suboptimal for Ubuntu and probably for many other projects as well, because the date you want depends on the version that the user is running. Indeed, if Launchpad had a way to record versions in which a bug was found as well, it would be possible to be much more accurate still about which bugs we show users: there's no point showing Hardy users a list of bugs that existed only for a couple of days in Lucid, at least not at the top of their list.

This general facility is called "version tracking" in Debian, and has been in debbugs since 2005 (http://lists.debian.org/debian-devel-announce/2005/07/msg00010.html). It was in the original design for Launchpad Bugs (under the name "infestations", which I never particularly liked ...), but was cut due, as I understand it, to time constraints. At Launchpad's current scale, I think it would be worth revisiting this decision; it is very useful when tracking multiple release series of software.

I agree that the version field is the hairiest part of the plan. It would be useful to be able to say, for example, that a security bug in Ubuntu is fixed in Ubuntu 8.04 LTS, 9.04, 9.10, and Lucid alpha 2 and later, or that a FooBarHum project bug is fixed in versions 2.2.1, 3.0.5, and trunk r775 and later. But any database schema smart enough to recognize those various version types would be quite a challenge, and I don't think solving that problem should be a prerequisite. Hypothetically, if you introduced version tracking later on, the free-form version info that had been entered for bugs fixed before version tracking was introduced would be no more problematic than the complete lack of version info for the older bugs fixed before the Fixed statuses were merged.

On Wed, Feb 10, 2010 at 06:13:53PM -0000, Matthew Paul Thomas wrote:
> I agree that the version field is the hairiest part of the plan. It
> would be useful to be able to say, for example, that a security bug in
> Ubuntu is fixed in Ubuntu 8.04 LTS, 9.04, 9.10, and Lucid alpha 2 and
> later, or that a FooBarHum project bug is fixed in versions 2.2.1,
> 3.0.5, and trunk r775 and later. But any database schema smart enough to
> recognize those various version types would be quite a challenge, and I
> don't think solving that problem should be a prerequisite.

I'm not sure I understand why this is a database schema problem. We
already have versions of projects and distribution source packages in
the database. Adding distribution versions ("8.04", "Lucid Alpha 2",
etc.) is certainly harder since in some cases those objects aren't in
the database at all, and I agree that there's no particular reason for
that to block the rest.

Fixes are (relatively) easy, anyway; you can use bzr bug revision
properties for lots of this, and then you just have to link *those* up
to versions, which in many cases we already have thanks to tags.
Information on where bugs were found is trickier, since you don't get
help from bzr.

The most challenging part of implementing version tracking in Debian was
representing the tree of available versions and figuring out how to
propagate found and fixed information across it. Launchpad quite often
has (and indeed draws) the project version tree already; it doesn't have
version trees for distribution source packages, but it has all the
changelog data so that could be reconstructed in the same way as we did
in Debian. The algorithm for propagating information across the tree is
now a solved problem, although it's possible that it would need
optimisation to work better in Launchpad's database model.

There are certainly UI problems, including how to deal with version
input, how to present the ability to adjust the found and fixed versions
recorded for a bug, and how (or whether!) to display information such as
"the version you're using suffers from this bug, but it was fixed over
here" versus "this bug has been fixed, but we don't know whether the
version you're using ever had it". In some cases it might make sense
for information only to be available via the API. debbugs is much more
heavily developer-targeted, so didn't need to solve these problems to
the same extent.

> Hypothetically, if you introduced version tracking later on, the free-
> form version info that had been entered for bugs fixed before version
> tracking was introduced would be no more problematic than the complete
> lack of version info for the older bugs fixed before the Fixed statuses
> were merged.

There is certainly no immediate problem introduced by not doing version
tracking, since we have no choice but to handle those problems in other
ways (inadequate though they often are). It's merely a big scary
problem that gets bigger and scarier the longer we leave it.

I do think it's appropriate to bring it up here, since it would be one
valid way to address the problem represented by this bug.

Changed status by mistake. Sorry

I feel that you should be able to set the bug status to 'FIX COMMITTED', but you should not be able to set it to 'FIX RELEASED'. Instead, it should automatically be changed to 'FIX RELEASED' when the milestone becomes inactive.

@Geoff, that would be inappropriate for bugs that are both introduced and fixed within a release, and would add massive overhead to projects that have users using interim versions (like Ubuntu itself - one of our largest users).

See also bug 754375, suggesting that lp:kanban should distinguish "needs release" from "released" by whether the task's has been released or not.

We may want to split out the various potential improvements from this bug: items 3 and 4 seem like genuine defects/improvements on their own.

Robert Collins: what do you mean by "items 3 and 4"? The changes proposed in comment #3 and comment #4 or something else?