Microphone input sounds horrible on ALSA with certain audio devices (like Intel)

Bug #900364 reported by Sean M. Pappalardo
40
This bug affects 8 people
Affects Status Importance Assigned to Milestone
Mixxx
Fix Released
Medium
Unassigned
1.10
Fix Released
Medium
Unassigned
1.11
Fix Released
Medium
Unassigned
2.0
Fix Released
Medium
Unassigned

Bug Description

On Debian Squeeze with the 2.6.32 and 3.2.0-rc4 kernels with an Intel audio chipset, the mic input sounds bad in Mixxx (fine in other software.) It happens at 44100Hz and 48000Hz with a USB headset mic and an analog one in the 1/8" mic jack,

Since I can record the bad sound, it's likely in the mic input code. See attachments.

00:1b.0 Audio device: Intel Corporation 5 Series/3400 Series Chipset High Definition Audio (rev 05)

Tags: microphone mic
Revision history for this message
Sean M. Pappalardo (pegasus-renegadetech) wrote :
Revision history for this message
Sean M. Pappalardo (pegasus-renegadetech) wrote :
Revision history for this message
RJ Skerry-Ryan (rryan) wrote :

Does this happen on other computers? I'd love to know how widespread it is. It works fine for me on my computers, but I'll test again before the release.

Revision history for this message
RJ Skerry-Ryan (rryan) wrote :

BTW, what sound API?

Revision history for this message
Sean M. Pappalardo (pegasus-renegadetech) wrote :

Haven't tested on other PCs yet. This happens with ALSA 1.0.23 but does _not_ happen with JACK.

summary: - Microphone input sounds horrible
+ Microphone input sounds horrible on ALSA
Revision history for this message
RJ Skerry-Ryan (rryan) wrote : Re: [Bug 900364] Re: Microphone input sounds horrible

Smells like a Mixxx SoundManager or a PortAudio bug then.

On Mon, Dec 5, 2011 at 3:14 PM, Sean M. Pappalardo <
<email address hidden>> wrote:

> Haven't tested on other PCs yet. This happens with ALSA 1.0.23 but does
> _not_ happen with JACK.
>
> --
> You received this bug notification because you are a member of Mixxx
> Development Team, which is subscribed to Mixxx.
> https://bugs.launchpad.net/bugs/900364
>
> Title:
> Microphone input sounds horrible
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/mixxx/+bug/900364/+subscriptions
>

Revision history for this message
RJ Skerry-Ryan (rryan) wrote : Re: Microphone input sounds horrible on ALSA

I know this goes against the evidence since it would mean it should apply to all Audio API, but do you have your VC gain set to something other than 1? It turns out that Vinyl Control gain is applied to the microphone input as well.

Revision history for this message
TiKaey (tikaey) wrote :

Happens here to. With build-in microphone and headset-microphone with ALSA, not With JACK.
The first try with Mixxx was the sound fine, but next time it was horrible.

Revision history for this message
Val (vk1266) wrote :

Same here. Choosing a different mic input somehow affects the degree of "horribleness" of the sound, but in the end the mic sound is still truly horrible.

Revision history for this message
5th_base (webtools-anders) wrote :

I have this problem as well, both in 1.10.1 and 1.11Beta2.
lspci info:
00:1b.0 Audio device: Intel Corporation 82801H (ICH8 Family) HD Audio Controller (rev 02)
Both USB and analog mic input is affected; USB mic volume pulsates at ~4-6Hz, analog sound sounds as if it's been resampled to at least one octave lower. Using a Dell Latitude D630 with Ubuntu 12.04 LTS.

Revision history for this message
Manolis Kapernaros (kapcom01) wrote :

Same problem here. Hp Probook 4530s, Ubuntu 13.04, Mixxx 1.10.1~dfsg0-1
My voice sounds like the Dracula.

Revision history for this message
RJ Skerry-Ryan (rryan) wrote :

Can one of you post a Mixxx log file from when this happens?

Revision history for this message
RJ Skerry-Ryan (rryan) wrote :

Also, does anything change when you change the Mixxx sample rate in Preferences -> Sound Hardware?

Revision history for this message
5th_base (webtools-anders) wrote :

I don't have my laptop available at the moment, but I do remember that changing the sample rate, or the latency, will change the microphone output sound.

Revision history for this message
Manolis Kapernaros (kapcom01) wrote :

Changing the sample rate didn't do any difference. Here is my log.

Revision history for this message
Manolis Kapernaros (kapcom01) wrote :

a sample recording (firtst with mixxx and then with audacity)

Revision history for this message
Manolis Kapernaros (kapcom01) wrote :

same proble with 1.11

Revision history for this message
Sean M. Pappalardo (pegasus-renegadetech) wrote :

Everyone, please also post your kernel version (uname -a) and sound hardware (lspci, just the line with your sound card.)

Revision history for this message
Manolis Kapernaros (kapcom01) wrote :

00:1b.0 Audio device: Intel Corporation 6 Series/C200 Series Chipset Family High Definition Audio Controller (rev 04)
Linux ProBook 3.8.0-22-generic #33-Ubuntu SMP Thu May 16 15:17:59 UTC 2013 i686 i686 i686 GNU/Linux

information type: Public → Public Security
information type: Public Security → Public
Revision history for this message
Val (vk1266) wrote :

Attached is a recording of me counting from one to ten.

Linux vkm2013 3.5.0-36-generic #57-Ubuntu SMP Wed Jun 19 15:10:49 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
00:1a.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #2 (rev 04)

Revision history for this message
Sean M. Pappalardo (pegasus-renegadetech) wrote :

Okay so this seems to be limited to Intel-based audio chip sets. Description changed.

description: updated
summary: - Microphone input sounds horrible on ALSA
+ Microphone input sounds horrible on ALSA on Intel audio device
summary: - Microphone input sounds horrible on ALSA on Intel audio device
+ Microphone input sounds horrible on ALSA with Intel audio devices
Revision history for this message
Slaker96 (slaker96) wrote : Re: Microphone input sounds horrible on ALSA with Intel audio devices

Hey guys,

Just wanted to add a note for ya'll to try. I have a laptop with Intel sound card also, Slackware 13.37 and Mixxx 1.11. Same issue, Mic sounds 2 octaves lower, til I selected the Microphone Input device "default". Not sysdefault or Intel HDA:.... just the option "default". Works great now, but I need to test more. I was happy to hear a working mic without the thought of using my behringer mixer!

I had it set to Microphone -> HDA Intel PCH: ALC269VB Analog (hw:0,0) and was experiencing the same issues as described above.

One thing though, in order to make it work after launching mixxx, I have to open mixxx preferences, and click ok to close. I don't have to change anything, just click preferences and ok.

I tested with 48000 and 44100 as well as 23ms and 11ms.

Revision history for this message
Val (vk1266) wrote :

I tried a possible workaround by Slaker96 in post #22 today. It did not work for me. However, in the process of trying the workaround I came across something that may get us closer to understanding the nature of this bug.

To operate Mixxx 1.11.0 on my laptop, I use an external USB audio card (Sabrent USB-SND8 Sound Card - 8 Channel 3D, 7.1 Surround Sound, USB 2.0, S-PDIF In/Out). Here is my audio config:

> aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: PCH [HDA Intel PCH], device 0: ALC269VC Analog [ALC269VC Analog]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: PCH [HDA Intel PCH], device 3: HDMI 0 [HDMI 0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: Device [USB Sound Device], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

> arecord -l
**** List of CAPTURE Hardware Devices ****
card 0: PCH [HDA Intel PCH], device 0: ALC269VC Analog [ALC269VC Analog]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: Device [USB Sound Device], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

In the Mixxx config:
Master Output is "USB Sound Device: USB Audio (hw:1.0)" (external audio card)
Headphones Output is "HDA Intel PCH: ALC269VC Analog (hw:0,0)" (internal audio card)
Microphone Input is "HDA Intel PCH: ALC269VC Analog (hw:0,0)" (internal audio card)

With this setup, microphone sound is indeed horrible as observed earlier by many. Now, set "Headphones Output" to "None". The microphone sound is good. Set the headphones output back to "HDA Intel PCH" and the microphone sound is again very bad.

When I set the microphone input to "USB Sound Device" and re-connect the mic to the external audio card, the microphone sound is again horrible.

It looks like the microphone sound is bad whenever the same hardware device is used as output for either the master audio or the headphones.

Any comments, Mixxx gurus?

Revision history for this message
Daniel Schürmann (daschuer) wrote :

Thank you Val for the worthy analysis.

in your case you suffer the same issue as Bug #1203249
I have lined out a solution there, but had no time to start with it.
But foe me this is a blocker bug for Mixxx 1.12

Revision history for this message
Val (vk1266) wrote :

Daniel,

Sorry it seems that Bug #1203249 may be unrelated to this one.

I could not recall hearing any noise with my headphones and I have just checked if I missed something. No, both my Master Audio on the external USB audio card and my headphone audio are near perfect playing simultaneously, there is no crackling and no noise on either of the outputs. Which is odd, considering the nature of Bug #1203249. For what it's worth, my external USB audio card is feeding digital audio over a Toslink cable to a 5.1 speaker system and my internal audio card sends analogue signal to headphones.

The "horrible microphone sound" issue appears to occur only when the microphone input device is set to the same audio device as one of the output devices.

Revision history for this message
Daniel Schürmann (daschuer) wrote :

Sorry for the misunderstanding.
Of cause microphone is something different then a headphone ;-)

But I assume that there is the same underlying root bug:
A race condition between the two audio callbacks issued from the two audio devices.

Revision history for this message
Sean M. Pappalardo (pegasus-renegadetech) wrote :

But the problem also occurs on a single audio device that has mic in and line out. It seems to be limited to Intel-based audio devices on Linux.

Revision history for this message
Val (vk1266) wrote :

Sean,

The problem seems to occur when microphone input device is set to the same device as one of the output devices. It may be so even if there is only one output device. In my tests, the mic sound went bad when I set the mic input to my external Sabrent audio card (note that the Sabrent card is NOT an Intel-based audio device). Certainly, I could not get rid of my internal Intel-based audio device, but based on my tests I do suspect that the problem may not be limited only to Intel-based audio devices.

Interestingly, the mic sound goes bad as soon as I set the mic input to the same device as one of the output devices in the configuration. I do not have to use that output device; a simple change in the configuration results in bad mic sound even if the output device is not used.

Revision history for this message
Val (vk1266) wrote :

Further to my earlier observations, I ran a few tests on a different computer and then some additional tests on the first computer. The attached PDF contains tabulated results of my tests on both computers.

In summary, it does indeed look like setting the same card for both playback and mic input triggers the problem with bad microphone sound. It does not matter if master audio or headphones is set for playback. It also does not matter if there is any playback in progress; simply setting one of the playback devices to the mic's card results in bad mic sound.

However, here is perhaps the most interesting part. Different audio cards exhibit different behaviour. The built-in HDA Intel cards on both computers I tested and the Sabrent USB audio card have this problem with bad mic sound. However, the Sound Blaster Sigma USB Headset does not have this problem. The mic sound is good even when the SB Sigma headset is used for both playback and mic input.

Notably, the problem is not limited to the Intel audio cards, despite the title of this bug. The Sabrent USB card I tested has this problem, too.

The test results in the attached PDF show that when the same card is used for playback and mic input, the mic sound is badly distorted. The cards have been colour-coded for easier reading of the table data: all cells where column heading and row heading contains the same colour, the sound is bad -- with the exception of SB Sigma USB card, which performs well in all circumstances.

I am also providing the detailed information on the hardware used in both systems in system1.txt and system2.txt. Since Launchpad does not allow multiple attachments, all three files have been packed into a zip archive.

Hopefully, this information will be helpful in chasing down and quashing this annoying bug.

Revision history for this message
Sean M. Pappalardo (pegasus-renegadetech) wrote :

Wow, excellent work Val!

On a whim, I just tested on my system again in Audacity and was able to nail down the problem! When I set Audacity to ALSA and mono (1-channel) recording from any mic source, the problem occurs. If I set it to stereo, it sounds fine. This is independent of the project sample rate. Val and others, can you please do this same test?

This may indeed be a PortAudio bug (Audacity uses it as well as Mixxx) in that it's interleaving the left and right audio samples when it should instead be throwing away every other one, but it seems more like a mis-communication between the sound hardware and PortAudio. E.g. PA tells the hardware to open just one audio channel but it still streams two regardless.

The ideal course of action would be to fix this in PortAudio, otherwise perhaps a workaround would be to offer an option in Mixxx to open the mic device as a stereo input.

summary: - Microphone input sounds horrible on ALSA with Intel audio devices
+ Microphone input sounds horrible on ALSA with certain audio devices
+ (like Intel)
Revision history for this message
Sean M. Pappalardo (pegasus-renegadetech) wrote :

I just sent a message to the PortAudio mailing list about this. We'll see what they say. As I did, I'm thinking the problem could be in ALSA itself too since we only see it with that API so I'll try to contact that list as well.

Revision history for this message
Val (vk1266) wrote :

Sean,

I am not observing what you have just described in Audacity. My Creative Live! webcam and the SB Sigma headset are both mono sources; as a result I cannot set them to stereo input in Audacity. When I record sound from either of them on mono input channel in Audacity, recordings sound just fine!

When I record mic input on HDA Intel, both mono and stereo recordings sound just fine!

Also... in the tests I described in my earlier message, the mic sound was good when a different card was used for playback. In other words, the mic sound was going bad as a result of setting mic input and playback to the same card and not as a result of changing mono/stereo setting. Am I missing something here?

Revision history for this message
Sean M. Pappalardo (pegasus-renegadetech) wrote :

No, I think you're right on. I may have just found a new problem with my system! XD Anyway, Alan Horstmann of PortAudio offered these suggestions, which I will do as I get some time:

"Is the trouble ALWAYS associated with a duplex stream - ie does the Mic always
record fine when there is no simultaneous output to the same device? It may
be that Portaudio is mis-handling this case, mono input with stereo output.

Look at un-compressed bad recording (perhaps with Audacity) - are the samples
obviously duplicated (each level twice)? Perhaps post a short wav (1 sec) or
a screen shot. The compressed files don't show this, but may be smoothing it
out.

Is Pulseaudio running on these systems? If so, try with that disabled
(pasuspender...) - in fact, for testing it is best to eliminate any effects
it may introduce unless it is implicated.

What Portaudio devices does this occur with? Is it only default/sysdefault,
or with the 'hw' devices?. If the hw ones, try setting the environment
variable PA_ALSA_PLUGHW to 1 before running the Mixx app in a terminal and
report if there is any change - that will enable channel adaption in Alsa.

Build the app with Portaudio debug enabled ( --enable-debug-output or define
PA_ENABLE_DEBUG_OUTPUT) and see the terminal output. Perhaps post it in
non-working case.

Build Portaudio stand-alone and run the test/example program pa_devs on the
troublesome system so we can see how the capabilities are reported.

There may be issues with how different Alsa soundcard drivers treat mono
microphone inputs. Some may output the same data to both channels, and
return twice as much data as expected. Pulseaudio, if operating, may also do
copying.

Hope this gives some avenues to pursue!"

Revision history for this message
Val (vk1266) wrote :

Sean: the mono/stereo issue you observed in Audacity may indeed be somehow related to this Mixxx issue. The audio card that performed well in my tests was the only audio card with mono Alsa input. All other audio cards in my tests had stereo Alsa input, and that's where the bad mic sound problem was observed. The detailed hardware info on my audio cards can be found in the linked zip file in post #29 above.

It may be just a coincidence. I have no other audio hardware to test this hypothesis. Hopefully, somebody else here will be able to test whether mono/stereo Alsa input is what makes the whole difference.

Revision history for this message
RJ Skerry-Ryan (rryan) wrote :

Clemens Ladisch pointed out that the ALSA hostapi in PortAudio will not respect your channel request if it is lower than the device's minimum number of channels. As a workaround, I prepped a patch that will just open the device in stereo even if we want to use it as mono and only use the first channel. Can somebody test?

Revision history for this message
Sean M. Pappalardo (pegasus-renegadetech) wrote :

I'll test, but you should too RJ on your previously-working devices to make sure those that do actually support mono aren't erroneously opened with stereo. (We should probably make this workaround conditional, with a checkbox near the mic input selection saying "open as stereo" with a tooltip that says "Choose this if your mic sound is corrupt" or similar.)

Revision history for this message
RJ Skerry-Ryan (rryan) wrote :

The patch is specific to ALSA and opens all devices in stereo. Because of the bug in PA it's impossible to tell when a device ought to have to been opened in stereo versus mono so the workaround is to open everything in stereo. This should be user-invisible (why have a checkbox that means "make it work properly"? and processing a mono input versus stereo isn't much of a savings computationally as we upmix mono channels to stereo anyway right when it hits the callback).

Revision history for this message
RJ Skerry-Ryan (rryan) wrote :

I committed the above workaround with a couple tweaks. Please test from master HEAD instead of using the patch.

Revision history for this message
Val (vk1266) wrote :

Well, success! Sort of...

The stereo mic device still shows up with two channels in the preferences dialog. In my case, leaving the default Channel 1 unchanged results in no improvement of the mic sound: it's still badly distorted. But when Channel 2 is selected, the mic sound is good!

Thank you RJ Ryan, Sean and everyone else who contributed. It looks like we are close to nailing this issue down.

Revision history for this message
RJ Skerry-Ryan (rryan) wrote : Re: [Bug 900364] Re: Microphone input sounds horrible on ALSA with certain audio devices (like Intel)

Val, can you confirm you were using Mixxx HEAD or were you using my patch
above?

On Tue, Oct 15, 2013 at 9:36 AM, Val <email address hidden> wrote:

> Well, success! Sort of...
>
> The stereo mic device still shows up with two channels in the
> preferences dialog. In my case, leaving the default Channel 1 unchanged
> results in no improvement of the mic sound: it's still badly distorted.
> But when Channel 2 is selected, the mic sound is good!
>
> Thank you RJ Ryan, Sean and everyone else who contributed. It looks like
> we are close to nailing this issue down.
>
> --
> You received this bug notification because you are a member of Mixxx
> Development Team, which is subscribed to Mixxx.
> https://bugs.launchpad.net/bugs/900364
>
> Title:
> Microphone input sounds horrible on ALSA with certain audio devices
> (like Intel)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/mixxx/+bug/900364/+subscriptions
>

Revision history for this message
Val (vk1266) wrote :

I used GitHub's "download ZIP" feature to obtain all sources in one zip
archive from https://github.com/mixxxdj/mixxx. It must have been the HEAD.

On 15 October 2013 10:03, RJ Ryan <email address hidden> wrote:

> Val, can you confirm you were using Mixxx HEAD or were you using my patch
> above?
>
>
> On Tue, Oct 15, 2013 at 9:36 AM, Val <email address hidden> wrote:
>
> > Well, success! Sort of...
> >
> > The stereo mic device still shows up with two channels in the
> > preferences dialog. In my case, leaving the default Channel 1 unchanged
> > results in no improvement of the mic sound: it's still badly distorted.
> > But when Channel 2 is selected, the mic sound is good!
> >
> > Thank you RJ Ryan, Sean and everyone else who contributed. It looks like
> > we are close to nailing this issue down.
> >
> > --
> > You received this bug notification because you are a member of Mixxx
> > Development Team, which is subscribed to Mixxx.
> > https://bugs.launchpad.net/bugs/900364
> >
> > Title:
> > Microphone input sounds horrible on ALSA with certain audio devices
> > (like Intel)
> >
> > To manage notifications about this bug go to:
> > https://bugs.launchpad.net/mixxx/+bug/900364/+subscriptions
> >
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/900364
>
> Title:
> Microphone input sounds horrible on ALSA with certain audio devices
> (like Intel)
>
> Status in Mixxx:
> Confirmed
> Status in Mixxx 1.10 series:
> Won't Fix
> Status in Mixxx 1.11 series:
> Confirmed
> Status in Mixxx 1.12 series:
> Confirmed
>
> Bug description:
> On Debian Squeeze with the 2.6.32 and 3.2.0-rc4 kernels with an Intel
> audio chipset, the mic input sounds bad in Mixxx (fine in other
> software.) It happens at 44100Hz and 48000Hz with a USB headset mic
> and an analog one in the 1/8" mic jack,
>
> Since I can record the bad sound, it's likely in the mic input code.
> See attachments.
>
> 00:1b.0 Audio device: Intel Corporation 5 Series/3400 Series Chipset
> High Definition Audio (rev 05)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/mixxx/+bug/900364/+subscriptions
>

Revision history for this message
Sean M. Pappalardo (pegasus-renegadetech) wrote :

OK, we have a patch (attached) from the PortAudio team that should take care of this problem. We need to test to see if it does indeed. (A change introduced in July 2010 did not correctly handle some stereo->mono cases; nothing Alsa-specific.)

Please test, those that are able, and report back here. (I won't have time in the next month or so.)

For reference, Alan Horstmann said:
"inside the buffer processing[:] When the formats do not differ (and there is no de-interleaving to do) the input conversion is skipped, and this thwarts the mechanism for input channel adaption, that uses the input data stride, since data is not then transferred. When the formats do differ, correct channel adaption occurs. It is possible that Alsa hw devices are the only users of this adaption mechanism, even though it is in the common code."

Revision history for this message
Owen Williams (ywwg) wrote :

Is it worth statically building/linking portaudio with Mixxx to fix this?

Revision history for this message
RJ Skerry-Ryan (rryan) wrote :

In the past we have published a custom libportaudio2 in our PPA to help
users w/ this (that was a much more drastic case though -- a PA bug caused
a complete Mixxx freeze when using Pulse). In this case I think if we can
find a workaround that fully works (Val mentioned my workaround didn't
fully work) then I think we can just ship the workaround until PA w/ fix is
widely distributed though the regular channels.

On Sun, Nov 10, 2013 at 3:09 PM, Owen Williams <email address hidden> wrote:

> Is it worth statically building/linking portaudio with Mixxx to fix
> this?
>
> --
> You received this bug notification because you are a member of Mixxx
> Development Team, which is subscribed to Mixxx.
> https://bugs.launchpad.net/bugs/900364
>
> Title:
> Microphone input sounds horrible on ALSA with certain audio devices
> (like Intel)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/mixxx/+bug/900364/+subscriptions
>

Revision history for this message
Val (vk1266) wrote :

@Sean, @RJ Ryan and others -- thank you for your work! -- I am intending to test the patch but this won't happen in the next two weeks, sorry.

Changed in mixxx:
status: Confirmed → In Progress
Revision history for this message
Sean M. Pappalardo (pegasus-renegadetech) wrote :

Okay, tests are successful for me! The patch Alan from the PA team created (now in PA trunk) takes care of the problem both with the latest revision of the 1.11 branch (r3881 at the time of writing) and version 1.11.0 as released (r3865.) So once that trickles down to the distros, we can finally put this to rest. Thanks to everyone for your help and patience!

Revision history for this message
Sean M. Pappalardo (pegasus-renegadetech) wrote :

(1.10 will automatically get this fix too once the distros ship the new PortAudio library)

Revision history for this message
RJ Skerry-Ryan (rryan) wrote :

Happy to report Trusty Tahr will have the latest PA snapshot which contains this fix:

http://packages.ubuntu.com/trusty/portaudio19-dev

Revision history for this message
RJ Skerry-Ryan (rryan) wrote :

Ok, I'm marking this fix released since no changes are needed in Mixxx. We have no control over when a particular distro will get this fix and it's already in Trusty so there's no more action for us to take.

Changed in mixxx:
status: In Progress → Fix Released
Revision history for this message
Swiftb0y (swiftb0y) wrote :

Mixxx now uses GitHub for bug tracking. This bug has been migrated to:
https://github.com/mixxxdj/mixxx/issues/6166

lock status: Metadata changes locked and limited to project staff
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.