apt fails if size of translated RFC822 message is much larger than the size of the original message.

screenshot2

screenshot3

Thanks for your report.

When this happens, at the exact same moment could you please run in a terminal:
$ fuser -v /var/cache/apt/archives/lock

That will list the processes holding the lock.

the command:

fuser -v /var/cache/apt/archives/lock

Nothing returns

I tried the following. Open synaptic. Reload packages. Upgrade packages. Apply and it works ok.

!!! Something important, if i select the package without searching it I can install it perfectly.

If I use the search button to find the same package, it appears the error described.

I'd like to find what is locking the directory anyway, could you please run (in a terminal) when this happens:
$ ps afx > /tmp/ps.txt

and attach the file ps.txt ?

I can reproduce it.
There is also an empty line in "NO AUTENTICADO"
It only occurs with LANG=es_ES.utf8 (cannot reproduce with french and english)

Yes, you are right.

Sorry for going back and forth but finally it's an apt bug and not a synaptic one. And a tricky one.
It is not specific to spanish language but spanish translators are a bit "verbose" and translation of the description of texlive-latex-extra is much larger than the buffer space allocated for it.

The record size of this package (t-l-e) is 34838 + 200 bytes for translated desc I suppose. That gives us a buffer size of 35837.
In spanish the record size for the same package is 38600. Much larger than the allocated 35837 bytes.

Since the parser doesn't reach the end of the message before reaching the size of the buffer it fails and returns false.
In pkgRecords::Lookup we do not catch this return status and continue as if nothing happened. I think that's what gives the funky error messages about unauthenticated packages and file locking.

TODO ?
- Catch the return code of the Jump method call in pkgRecords::Lookup and do what ?
- Increase the static buffer size in the debRecordParser constructor significantly ?
- Dynamically resize the buffer size in pkgTagSection::Scan ?

Michael what's your ideas ?

and all of it is triggered by the call to pkg->summary() and pkg->description() when doing a normal search in synaptic.

it's late and my brain is going slow: 34838 + 200 = 35038

 ps afx > /tmp/ps.txt

  PID TTY STAT TIME COMMAND
    2 ? S 0:00 [kthreadd]
    3 ? S 0:00 \_ [migration/0]
    4 ? S 0:04 \_ [ksoftirqd/0]
    5 ? S 0:00 \_ [watchdog/0]
    6 ? S 0:01 \_ [migration/1]
    7 ? S 0:04 \_ [ksoftirqd/1]
    8 ? S 0:00 \_ [watchdog/1]
    9 ? S 0:00 \_ [events/0]
   10 ? S 0:00 \_ [events/1]
   11 ? S 0:00 \_ [cpuset]
   12 ? S 0:00 \_ [khelper]
   13 ? S 0:00 \_ [netns]
   14 ? S 0:00 \_ [async/mgr]
   15 ? S 0:00 \_ [pm]
   16 ? S 0:03 \_ [sync_supers]
   18 ? S 0:00 \_ [bdi-default]
   19 ? S 0:00 \_ [kintegrityd/0]
   20 ? S 0:00 \_ [kintegrityd/1]
   21 ? S 0:00 \_ [kblockd/0]
   22 ? S 0:00 \_ [kblockd/1]
   23 ? S 0:00 \_ [kacpid]
   24 ? S 0:00 \_ [kacpi_notify]
   25 ? S 0:00 \_ [kacpi_hotplug]
   26 ? S 0:08 \_ [ata/0]
   27 ? S 0:01 \_ [ata/1]
   28 ? S 0:00 \_ [ata_aux]
   29 ? S 0:00 \_ [ksuspend_usbd]
   30 ? S 0:00 \_ [khubd]
   31 ? S 0:00 \_ [kseriod]
   32 ? S 0:00 \_ [kmmcd]
   35 ? S 0:00 \_ [khungtaskd]
   36 ? S 0:16 \_ [kswapd0]
   37 ? SN 0:00 \_ [ksmd]
   38 ? S 0:00 \_ [aio/0]
   39 ? S 0:00 \_ [aio/1]
   40 ? S 0:00 \_ [ecryptfs-kthrea]
   41 ? S 0:00 \_ [crypto/0]
   42 ? S 0:00 \_ [crypto/1]
   46 ? S 0:20 \_ [scsi_eh_0]
   47 ? S 0:00 \_ [scsi_eh_1]
   50 ? S 0:00 \_ [kstriped]
   51 ? S 0:00 \_ [kmpathd/0]
   52 ? S 0:00 \_ [kmpathd/1]
   53 ? S 0:00 \_ [kmpath_handlerd]
   54 ? S 0:00 \_ [ksnapd]
   55 ? S 0:42 \_ [kondemand/0]
   56 ? S 0:15 \_ [kondemand/1]
   57 ? S 0:00 \_ [kconservative/0]
   58 ? S 0:00 \_ [kconservative/1]
  247 ? S 0:00 \_ [scsi_eh_2]
  248 ? S 0:00 \_ [scsi_eh_3]
  249 ? S 0:00 \_ [scsi_eh_4]
  250 ? S 0:00 \_ [scsi_eh_5]
  251 ? S 0:00 \_ [khpsbpkt]
  254 ? S 0:00 \_ [scsi_eh_6]
  255 ? S 0:28 \_ [usb-storage]
  259 ? S 0:00 \_ [scsi_eh_7]
  260 ? S 0:00 \_ [usb-storage]
  266 ? S 0:00 \_ [knodemgrd_0]
  278 ? S 0:00 \_ [reiserfs/0]
  279 ? S 0:00 \_ [reiserfs/1]
  281 ? S 0:00 \_ [usbhid_resumer]
  315 ? S 0:02 \_ [flush-8:0]
  717 ? S 0:00 \_ [kpsmoused]
  781 ? S 0:00 \_ [hd-audio0]
22774 ? S 0:00 \_ [bluetooth]
22778 ? S< 0:00 \_ [krfcommd]
    1 ? Ss 0:00 /sbin/init
  349 ? S 0:00 upstart-udev-bridge --daemon
  352 ? S<s 0:00 udevd --daemon
29585 ? S< 0:00 \_ udevd --daemon
22777 ? S< 0:00 \_ udevd --daemon
  729 ? Sl 8:29 ...

Thanks for your comment but that's ok. The source of the problem is located and a fix will be provided.

Lucid was released and this bug is still annoying everybody...

... everybody using some languages among which spanish, doing a search and then trying to install a package.
There are still plenty of bugs to fix despite Lucid was released, so feel free to help and provide few patches. You're welcome.

As a member of the spanish translation group I would like to know if there is anything we can do to workaround this issue while a patch is developed. This bug is affecting a lot of spanish users indeed.

Thanks for helping Fernando, I'd like to get maintainer's insight before doing anything. He knows the arcane of apt and what is the best solution.
The bug is triggered by the size of the spanish description of texlive-latex-extra, so a possible workaround would be to limit this translation to something around 35K characters. But it will fix only this translation and only for spanish and this bug may happen for other languages too and it's lot of work to adapt translations to work around that kind of thing.

About my previous comment:
it also hits people doing
$ LANG=es_ES.utf8 apt-cache show texlive-latex-extra

This is a wider audience than only synaptic users.

Thanks for the bugreport and thanks to Jean-Baptiste Lallement for analyzing the problem.

The bug here is the following bit in debrecords.cc:

                  Tags(&File,Cache.Head().MaxVerFileSize + 200)

MaxVerFileSize is the maximum size for a package record found in the Packages file. This is not sufficient
in the days of Translations-$lang and we need to update the code so that it takes the translation files into
consideration for the maximum size too. This bug has not shown yet because the code already uses the
biggest record. In this case that was probably t-l-e already and even with the additional meta-data from
the packages file (like dependencies etc) it is not enough.

I see two possible fixes:

Simple:
- Tags(&File,Cache.Head().MaxVerFileSize + 200)
+ Tags(&File,Cache.Head().MaxVerFileSize * 2)

Better:
- Tags(&File,Cache.Head().MaxVerFileSize + 200)
+ Tags(&File, max(Cache.Head().MaxVerFileSize,Cache.Head().MaxDescFileSize) + 200)

I've just applied the patch and everything seems to be working for me too.

How to apply the patch.

Thanks Michael,
I recompiled libapt-pkg-libc6.10-6.so.4.8.0 with your patch and solved this bug.

Copy the attached library to /usr/lib/libapt-pkg-libc6.10-6.so.4.8.0 to solve the problem while an updated apt package arrive.
I tried to make the deb but I failed becaus "make install" don't work.

Sorry, previous upload was corrupt, I reattached the file ONLY for 64 bits!

How to:

sudo mv /usr/lib/libapt-pkg-libc6.10-6.so.4.8.0 /usr/lib/libapt-pkg-libc6.10-6.so.4.8.0.copia
sudo tar zvxf libapt-pkg.tar.gz -C /usr/lib/

Greetz

finally the 32 bits version.

Thanks for your help, gracias Dept.Técnico (Extreme Micro S.L.).The patch works.

Accepted apt into lucid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

I tested the package at lucid-proposed and works. At least in lucid 32 bits.

Thanks

The package works for me. Thanks for your help.

Hi, sorry for being such a noob, but i had the bug you are commenting and trying to fix it i did the following:

sudo mv /usr/lib/libapt-pkg-libc6.10-6.so.4.8.0 /usr/lib/libapt-pkg-libc6.10-6.so.4.8.0.copia
sudo tar zvxf libapt-pkg.tar.gz -C /usr/lib/

but WITHOUT DOWNLOADING THE FILE! (i didnt see it!, i thought the bug would be fix with those two lines)

Now i cannot open synaptic, nor the update manager...and i have a red error sign in the upper panel saying there is an error while triyng to check for updates.

What do you suggest me to do? how can i undo whatever i did with those commands?

Thanks in advance!

Neither can open the software center...

Should i reinstall? or there is somethin i can do?

@gakna: try the following command:
sudo mv /usr/lib/libapt-pkg-libc6.10-6.so.4.8.0.copia /usr/lib/libapt-pkg-libc6.10-6.so.4.8.0

It should restore the backup you made to its original name.

Hi Jean Baptiste, thanks for your answer.

I pasted the command and it didnt work.

I think that, besides copying the command (which i did first):

sudo mv /usr/lib/libapt-pkg-libc6.10-6.so.4.8.0 /usr/lib/libapt-pkg-libc6.10-6.so.4.8.0.copia
sudo tar zvxf libapt-pkg.tar.gz -C /usr/lib/

later i pasted the file that i downloaded from here (the patch i guess) in the directory /usr/lib REPLACING the original one (i did it opening nautilus as sudo to be able to paste it in the folder).

The situation is still the same, i cant open synaptic or the update manager nor the software center and the error sign is still there....

Anything i can do? reinstall maybe?

This appears as the error in the terminal:

apt-get: error while loading shared libraries: libapt-pkg-libc6.10-6.so.4.8: cannot open shared object file: No such file or directory

Apparently i fixed it, i just renamed the file libapt-pkg-libc6.10-6.so.4.8.0 taking out the last zero, and everything came to life...and the original bug is fixed...

is it ok what i did or it will bring some other problems?

Thanks in advance!!

This bug was fixed in the package apt - 0.7.25.3ubuntu8

---------------
apt (0.7.25.3ubuntu8) lucid-proposed; urgency=low

  [ Loïc Minier ]
  * Use https:// in Vcs-Bzr URL.

  [ Michael Vogt ]
  * apt-pkg/deb/debrecords.cc:
    - fix max tag buffer size (LP: #545336, closes: #578959)
  * apt-pkg/indexfile.cc:
    - If no "_" is found in the language code, try to find a "."
      This is required for languages like Esperanto that have no
      county associated with them (LP: #560956)
      Thanks to "Aisano" for the fix
 -- Michael Vogt <email address hidden> Wed, 05 May 2010 10:33:46 +0200

Copied to maverick.