[upstream] Arabic text gets deformed when creating a PDF in LibreOffice Writer

Description:
Creating a PDF from a document written in the Arabic script deforms the textual content of the document, although it looks fine on the screen.

For example, see the attached PDF created with Writer 5.4.6.2 on Ubuntu 17.10, where the example sentence "اشترى بلال خمسة آلاف كتاب وَأَنَا اشْتَرَيْتُهَا مِنْهُ" looks as it should, but when you view it with any PDF reader, such as evince, copying the text deforms most of the words. Some characters are clearly visible but cannot be selected or searched (such as ى at the end of the first word اشترى). If I search for the second word بلال, evince tells me there are no matches in the document. The same happens when converting the file with pdftotext, which produces the following output:

‫اشتر‬

‫للا مسة لفا كتاب وَأَنَا ْ‬
‫ه‬
‫اشت َ َريْتُهَا ِ‬
‫من ْ ُ‬

Here only two of the eight words are intact, the rest are garbled in one way or another. If the text is in Latin script, both evince and pdftotext behave as expected, meaning that the textual content is transferred correctly from Writer to the PDF.

On LO 6.0.3.2 on Ubuntu 18.04, the textual content is preserved a little better but it is still quite garbled. This is the output from pdftotext:

ه‬
اشترى للا خمسة آفا كتاب وَأنَا اشْ ت َ َريْتُهَا ِ‬
من ْ ُ‬

Here four out of the eight words are intact, and for example the last word of the sentence is divided so that the last full character is found on the first line and the rest on the third line. Some diacritics are found where they are supposed to be, some others not.

MS Word 2007 handles this case better, although it's not perfect either. This is the output from pdftotext:

اشترى بالل خمسة آالف كتاب وأنا اشتريتها منه‬

Here all diacritics are dropped and all sequences of ل (U+0644) + ا (U+0627) are reversed turning لا into ال. Otherwise the sentence is intact.

This bug was first reported on Launchpad for LO 5.4.6.2 on Ubuntu 17.10 at: https://bugs.launchpad.net/ubuntu/+source/libreoffice/+bug/1772439 . After my initial report, I have upgraded to LO 6.0.3.2 where the problem persists, although the actual output is different. Another user on Launchpad confirmed the bug on LO 6.0.3.2, as well.

Steps to Reproduce:
1. In a new Writer document, type some text in Arabic. My example sentence was: اشترى بلال خمسة آلاف كتاب وَأَنَا اشْتَرَيْتُهَا مِنْهُ
2. Create a PDF.
3. Open the created PDF with a PDF reader (such as evince) and type one of the words in the Search dialog, e.g. بلال. Alternatively select the word in the PDF reader and copy-paste it somewhere else. You can also convert the PDF to text using a utility like pdftotext.

Actual Results:
The PDF reader reports there are no matches for some of the words in the document, although they are all clearly visible. Selecting and copy-pasting the word garbles it. Pdftotext's output is garbled.

Expected Results:
All the words that are visible should also be searchable in a PDF reader, copy-pasting should preserve the text, and the output of pdftotext should match the original document.

Reproducible: Always

User Profile Reset: No

Additional Info:

Created attachment 144554
PDF created with LO 5.4.6.2 where textual content is garbled

Repro. Can only successfully search with individual glyphs in PDF

Arch Linux 64-bit
Version: 6.2.0.0.alpha0+
Build ID: 8b1501d80dc9d3f42c351c6e026fa737e116cae5
CPU threads: 8; OS: Linux 4.18; UI render: default; VCL: gtk3_kde5;
Locale: fi-FI (fi_FI.UTF-8); Calc: threaded
Built on 23 September 2018

Dear vaaydayaasra,

To make sure we're focusing on the bugs that affect our users today, LibreOffice QA is asking bug reporters and confirmers to retest open, confirmed bugs which have not been touched for over a year.

There have been thousands of bug fixes and commits since anyone checked on this bug report. During that time, it's possible that the bug has been fixed, or the details of the problem have changed. We'd really appreciate your help in getting confirmation that the bug is still present.

If you have time, please do the following:

Test to see if the bug is still present with the latest version of LibreOffice from https://www.libreoffice.org/download/

If the bug is present, please leave a comment that includes the information from Help - About LibreOffice.

If the bug is NOT present, please set the bug's Status field to RESOLVED-WORKSFORME and leave a comment that includes the information from Help - About LibreOffice.

Please DO NOT

Update the version field
Reply via email (please reply directly on the bug tracker)
Set the bug's Status field to RESOLVED - FIXED (this status has a particular meaning that is not
appropriate in this case)

If you want to do more to help you can test to see if your issue is a REGRESSION. To do so:
1. Download and install oldest version of LibreOffice (usually 3.3 unless your bug pertains to a feature added after 3.3) from http://downloadarchive.documentfoundation.org/libreoffice/old/

2. Test your bug
3. Leave a comment with your results.
4a. If the bug was present with 3.3 - set version to 'inherited from OOo';
4b. If the bug was not present in 3.3 - add 'regression' to keyword

Feel free to come ask questions or to say hello in our QA chat: https://kiwiirc.com/nextclient/irc.freenode.net/#libreoffice-qa

Thank you for helping us make LibreOffice even better for everyone!

Warm Regards,
QA Team

MassPing-UntouchedBug

Still reproducible on:

Version: 6.3.2.2
Build ID: libreoffice-6.3.2.2-snap1
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3;
Locale: fi-FI (fi_FI.UTF-8); UI-Language: en-US
Calc: threaded

pdftotext's output is again different from my initial report but it's still garbled:

أ‬
ه‬
ن‬
ا م‬
ه‬
ت‬
ي‬
ر‬
ا اشْت‬
و ن‬
اشترى بالل خمسة آالف كتاب َ‬

This time the beginning of the sentence (found on the last line of the output) is already quite good, though ل and ا in the ligature لا are reversed. Thus on evince بالل matches بلال. The end of the sentence where there are diacritical vowel marks is worse than in my initial report.

Dear vaaydayaasra,

To make sure we're focusing on the bugs that affect our users today, LibreOffice QA is asking bug reporters and confirmers to retest open, confirmed bugs which have not been touched for over a year.

There have been thousands of bug fixes and commits since anyone checked on this bug report. During that time, it's possible that the bug has been fixed, or the details of the problem have changed. We'd really appreciate your help in getting confirmation that the bug is still present.

If you have time, please do the following:

Test to see if the bug is still present with the latest version of LibreOffice from https://www.libreoffice.org/download/

If the bug is present, please leave a comment that includes the information from Help - About LibreOffice.

If the bug is NOT present, please set the bug's Status field to RESOLVED-WORKSFORME and leave a comment that includes the information from Help - About LibreOffice.

Please DO NOT

Update the version field
Reply via email (please reply directly on the bug tracker)
Set the bug's Status field to RESOLVED - FIXED (this status has a particular meaning that is not
appropriate in this case)

If you want to do more to help you can test to see if your issue is a REGRESSION. To do so:
1. Download and install oldest version of LibreOffice (usually 3.3 unless your bug pertains to a feature added after 3.3) from https://downloadarchive.documentfoundation.org/libreoffice/old/

2. Test your bug
3. Leave a comment with your results.
4a. If the bug was present with 3.3 - set version to 'inherited from OOo';
4b. If the bug was not present in 3.3 - add 'regression' to keyword

Feel free to come ask questions or to say hello in our QA chat: https://kiwiirc.com/nextclient/irc.freenode.net/#libreoffice-qa

Thank you for helping us make LibreOffice even better for everyone!

Warm Regards,
QA Team

MassPing-UntouchedBug

The problem seems to have been resolved on LO 7.3.0.3 on Windows 10. To test PDF output this time, I used Adobe Acrobat DC 2021.011.20039 64-bit. I haven't tested on Linux, where the problem initially appeared.

Version: 7.3.0.3 (x64) / LibreOffice Community
Build ID: 0f246aa12d0eee4a0f7adcefbf7c878fc2238db3
CPU threads: 4; OS: Windows 10.0 Build 19044; UI render: Skia/Raster; VCL: win
Locale: fr-FR (fr_FR); UI: fr-FR
Calc: CL

Unfortunately still reproduced on Linux

Arch Linux 64-bit
Version: 7.4.0.0.alpha0+ / LibreOffice Community
Build ID: 8f2b1b1cb84e1ae3139eb90b8efdf61e608adbad
CPU threads: 8; OS: Linux 5.16; UI render: default; VCL: kf5 (cairo+xcb)
Locale: fi-FI (fi_FI.UTF-8); UI: en-US
Calc: threaded Jumbo
Built on 24 February 2022