Node body contents displayed is unpredictably incorrect

On Wed, 11 Apr 2012 17:15:43 -0000
SegundoBob <email address hidden> wrote:

> I continued using Leo-Editor and looked back at the node several
> times. Eventually, the node showed all 8,000 words, but I don't know
> what made them finally show up. I can't repeat this problem.

Sounds somewhat like
http://groups.google.com/group/leo-editor/browse_thread/thread/af340567c672290d?fwc=1

Cheers -Terry

On Wed, Apr 11, 2012 at 12:15 PM, SegundoBob <email address hidden> wrote:
> Public bug reported:
>
> A Leo-Editor Python script pasted about 8,000 words from the clipboard
> to the body of a new node that the script created.  When I looked at the
> body of the new node, it contained much less than the 8,000 words, but
> the script correctly reported that the body should contain about 8,000
> words.  I continued using Leo-Editor and looked back at the node several
> times.  Eventually, the node showed all 8,000 words, but I don't know
> what made them finally show up.  I can't repeat this problem.

Interesting timing of your bug report. Last night I started work on
bug 971166: Prints to tabs in the Log Pane are UTF-8 encoded
https://bugs.launchpad.net/leo-editor/+bug/971166

Despite the title of the bug, I believe this bug is related to Leo's
Qt paste code, for the following two reasons.

1. The workaround for bug 971166 is to do the following:

A. Paste the text into the body text of a Leo node, say node N.
B. Move *away* from node N, then move back to N.
C. Copy the body text of node N.

Now the text is as expected, and can be pasted in SciTe or any other editor.

I suspect that doing something similar was what restored the body text
in your situation.

2. The bug happens on Python 3.2, in which case everything is always a
unicode string and g.toUnicode never does anything. Thus, (imo) the
(unicode) text encoding is not likely the problem. But as I discuss
below, *another* encoding of the text is likely the problem.

My model for what is happening is as follows. The leoFrame.pasteText
code calls::

    w.insert(i,s)
    w.see(i+len(s) + 2)

Here w is a leoQTextWidget, a subclass of QTextBrowser. The
corresponding leoQTextWidget.insert code is, in essence::

    cursor = w.textCursor()
    cursor.setPosition(i)
    cursor.insertText(s) # This cause an incremental call to recolor.

The comment indicates that this is an optimization so that we can do a
partial recolor.

So there is something different between using this code and using the
setAllText code, which is basically::

    w.setPlainText(s)

Once again, w is a leoQTExtWidget.

I suspect that there is an *internal* encoding difference between
Leo's body pane and Leo's log pane. Indeed, Leo writes to log panes
with log.put. Here is the "business end" of log.put::

        s=s.replace('&', '&amp;').replace('<', '&lt;').replace('>', '&gt;')
        if not self.wrap:
            s = s.replace(' ','&nbsp;')
        s = s.rstrip().replace('\n','<br>')
        s = '<font color="%s">%s</font>' % (color,s)
        if 1: # The original way.
            w.append(s)
        else: # This also works.
            w.insertHtml(s+'<br>')

So internally, the log pane contains html. Somehow this is causing
confusion when the text is copied and pasted into the body pane, which
is *not* represented as html. Indeed, the body pane must be plain
text in order to be colorized.

The bottom line is that *Leo's* model for what the text should be is
correct, but somehow (temporarily) Qt's model for what the text should
be is incorrect.

A possible fix would be to use the setAllText way, at least for
p...

Fixed at the trunk at rev 5247. Here is the checkin log:

Fixed two bugs: 971166 and 979142. Fixes to leoFrame.pasteText and leoQTextEditWidget.get.

The "optimized" get method fails (somehow) when mixing html-encoded and plain-encoded text.

- 971166: Prints to tabs in the Log Pane are UTF-8 encoded

- 979142: Node body contents displayed is unpredictably incorrect

All unit tests pass with Python 2.x and 3.x.

A few minor changes:

- Simplified g.es code just slightly.

- Added disabled trace to leoQtLog.put.

- Added disambiguating headline comment to codewise Unicode Utils node.