add-apt-repository will write duplicate entries to /etc/apt/sources.list

I was able to confirm this using Oneiric. Thanks for providing detailed steps to recreate this bug.

I found that add-apt-repository calls

 SoftwareProperties().add_source_from_line(line)

In /usr/share/pyshared/softwareproperties/SoftwareProperties.py I found

 self.sourceslist.list.append(new_deb_entry)
 self.sourceslist.list.append(new_debsrc_entry)

I think it would be better to call self.sourceslist.add(), since it avoids
duplicates. The add() function requires more arguments, but the only ones
I had to hard-code were elsewhere in SoftwareProperties.py anyway.

I've included a patch (against 0.80.9 through 0.82) which seems to make thing
work nicely. I hard-coded deb, deb-src, and main.

The attachment "patch to SoftwareProperties.py to make add_source_from_line idempotent" of this bug report has been identified as being a patch. The ubuntu-reviewers team has been subscribed to the bug report so that they can review the patch. In the event that this is in fact not a patch you can resolve this situation by removing the tag 'patch' from the bug report and editing the attachment so that it is not flagged as a patch. Additionally, if you are member of the ubuntu-sponsors please also unsubscribe the team from this bug report.

[This is an automated message performed by a Launchpad user owned by Brian Murray. Please contact him regarding any issues with the action taken in this bug report.]

Thanks for the patch Nick!

While your use of self.sourceslist.add() seems good, I don't like how it is using similar logic that is in ppa.py in SoftwareProperties.py's add_source_from_line(). Wouldn't it be better to extract out the parts of new_deb_entry and new_debsrc_entry and give those parts to self.sourceslist.add()?

NAK pending comments from submitter. Please resubscribe ubuntu-sponsors once you have commented and/or upddated your patch. Thanks again!

Thanks Jamie,
Does this new patch address your concerns? Instead of hard-coding the type, distro, components, etc. it extracts them back out of the SourceEntry object.
Nick

Does it check ^#deb lines?

Nick Russo - thanks for the patch. I've merged it with a couple of minor changes.

This bug was fixed in the package software-properties - 0.82.3

---------------
software-properties (0.82.3) precise; urgency=low

  [ Brian Murray ]
  * when adding new repositories use sourceslist.add instead of append thereby
    preventing duplicate entires. Thanks to Nick Russo for the patch.
    LP: #854841

  [ Robert Roth ]
  * Updated expand properties to properly expand the bottom component to avoid
    putting empty space between components. LP: #912557
  * Added symbolic link for add-apt-repository manpage under the
    apt-add-repository name, LP: #620098
  * Changed Revert button mnemonic to avoid collision with Remove, LP: #652523
  * Handle URLError from ppa pages, instruct the user to check the internet
    connection (LP: #502698)

  [ Marc Deslauriers ]
  * SECURITY UPDATE: incorrect ssl certificate validation (LP: #915210)
    - softwareproperties/ppa.py: use pycurl to download the signing key
      fingerprint.
    - tests/test_lp.py: add test.
    - debian/control: add python-pycurl dependency.
    - CVE-2011-4407
  * Wait for PPA GPG key to get imported before ending thread (LP: #888417)
 -- Marc Deslauriers <email address hidden> Fri, 03 Feb 2012 08:16:22 -0500

This bug is NOT fixed yet!

I am getting:

W: Duplicate sources.list entry http://ppa.launchpad.net/xxxxx oneiric/main amd64 Packages (/var/lib/apt/lists/ppa.launchpad.net_xxxxx_dists_oneiric_main_binary-amd64_Packages)
W: Duplicate sources.list entry http://ppa.launchpad.net/xxxxx/ubuntu/ oneiric/main i386 Packages (/var/lib/apt/lists/ppa.launchpad.net_xxxxx_dists_oneiric_main_binary-i386_Packages)
W: You may want to run apt-get update to correct these problems

After running sudo apt-get update, this problem is not resolved.

lsb_release -a
LSB Version: core-2.0-amd64:core-2.0-noarch:core-3.0-amd64:core-3.0-noarch:core-3.1-amd64:core-3.1-noarch:core-3.2-amd64:core-3.2-noarch:core-4.0-amd64:core-4.0-noarch
Distributor ID: Ubuntu
Description: Ubuntu 11.10
Release: 11.10
Codename: oneiric

grep -Ri launchpad* /etc/apt/sources.list
deb http://ppa.launchpad.net/steubens/transmission-proxy-support/ubuntu natty main
deb-src http://ppa.launchpad.net/steubens/transmission-proxy-support/ubuntu natty main
deb http://ppa.launchpad.net/gnome3-team/gnome3/ubuntu oneiric main
deb-src http://ppa.launchpad.net/gnome3-team/gnome3/ubuntu oneiric main
(seems like some are not listed here)

Corrective measure needed:
Implement a check to see if any PPA was already added prior to adding it to the system.

@John Doe - you'll notice in comment 9 the following:

This bug was fixed in the package software-properties - 0.82.3

---------------
software-properties (0.82.3) precise; urgency=low

This bug was fixed, just for the precise (12.04), release of Ubuntu - not for 11.10 which you are running.

@Brian Murray - This still seems to be a problem in trusty (14.04).

Bump. This still exists.

Could you add a test case for this? I only see a duplicated commented out deb-src entry which doesn't seem terribly important.

Disregarding my typo of partner this was my test:

(trusty-amd64)root@impulse:/home/bdmurray# add-apt-repository "deb http://archive.canonical.com/ubuntu trusty partnet"
(trusty-amd64)root@impulse:/home/bdmurray# cat /etc/apt/sources.list
deb http://ubuntu.osuosl.org/ubuntu trusty main restricted universe multiverse
deb-src http://ubuntu.osuosl.org/ubuntu trusty main restricted universe multiverse
deb http://ubuntu.osuosl.org/ubuntu trusty-updates main restricted universe multiverse
deb-src http://ubuntu.osuosl.org/ubuntu trusty-updates main restricted universe multiverse
deb http://ubuntu.osuosl.org/ubuntu trusty-security main restricted universe multiverse
deb http://archive.canonical.com/ubuntu trusty partnet
# deb-src http://archive.canonical.com/ubuntu trusty partnet
deb-src http://ubuntu.osuosl.org/ubuntu trusty-security main restricted universe multiverse
(trusty-amd64)root@impulse:/home/bdmurray# add-apt-repository "deb http://archive.canonical.com/ubuntu trusty partnet"
(trusty-amd64)root@impulse:/home/bdmurray# cat /etc/apt/sources.list
deb http://ubuntu.osuosl.org/ubuntu trusty main restricted universe multiverse
deb-src http://ubuntu.osuosl.org/ubuntu trusty main restricted universe multiverse
deb http://ubuntu.osuosl.org/ubuntu trusty-updates main restricted universe multiverse
deb-src http://ubuntu.osuosl.org/ubuntu trusty-updates main restricted universe multiverse
deb http://ubuntu.osuosl.org/ubuntu trusty-security main restricted universe multiverse
deb http://archive.canonical.com/ubuntu trusty partnet
# deb-src http://archive.canonical.com/ubuntu trusty partnet
# deb-src http://archive.canonical.com/ubuntu trusty partnet
deb-src http://ubuntu.osuosl.org/ubuntu trusty-security main restricted universe multiverse