bound method ViewRenderedController.update

On Tue, Jan 10, 2012 at 5:34 AM, Lewis Neal <email address hidden> wrote:
> Public bug reported:

>    s = publish_string(s,writer_name='html')
>  File "C:\Python32\lib\site-packages\docutils\core.py", line 400, in publish_string
>    enable_exit_status=enable_exit_status)
>  File "C:\Python32\lib\site-packages\docutils\core.py", line 648, in publish_programmatically
>    output = pub.publish(enable_exit_status=enable_exit_status)
>  File "C:\Python32\lib\site-packages\docutils\core.py", line 232, in publish
>    sys.exit(exit_status)
> SystemExit: 1

Thanks for this report. Let's hope there is a way to dissuade
docutils from calling sys.exit on minor errors ;-) Looks like the
enable_exit_status flag might do the trick.

In the meantime, one workaround for you would be to hack the docutils
sources. I've resorted to that in the past...

Edward

On Tue, Jan 10, 2012 at 7:44 AM, Edward K. Ream <email address hidden> wrote:

> Thanks for this report.  Let's hope there is a way to dissuade
> docutils from calling sys.exit on minor errors ;-)  Looks like the
> enable_exit_status flag might do the trick.

After looking at the code it appears that clearing the
enable_exit_status flag would not help, even if it were possible.

A more promising approach would be to call docutils only if the
various files that it will be using actually exist. I'll attempt a
fix later today.

Edward

On Tue, Jan 10, 2012 at 9:07 AM, Edward K. Ream <email address hidden> wrote:

> A more promising approach would be to call docutils only if the
> various files that it will be using actually exist.  I'll attempt a
> fix later today.

After re-reading the traceback, this appears that this is some kind of
installation problem with docutils. The relevant line from the
traceback is::

  IOError: [Errno 2] No such file or directory:
  '../../../../../lib/site-packages/docutils/writers/html4css1/html4css1.css'

On my installation, the file
site-packages/docutils/writers/html4css1/html4css1.css does indeed
exist. docutils should use default settings if this file goes
missing, so this does not seem like Leo's problem. For this reason, I
am inclined not to fix this bug.

Edward

I'm not sure it's an installation problem as the /lib/site-packages/docutils/writers/html4css1/html4css1.css1 file is present on my system.
The traceback occurred initially on my leo file and then again on an edited copy to focus on the reSt node. However after the first 2 tracebacks the alt-x vr command then proceeded to work successfully; I did not change anything on my system!
Either docutils or leo is looking for html4css1.css1 but gives up too quickly? But why does it find the file with subsequent attempts? I tried many times to reproduce the behaviour without success.

The docutils installed is the latest snapshot.
As you seem familiar with the sys.exit problem do you see any value in reporting this to the docutils maintainers as a bug?

Thanks
Lewis

On Tue, 10 Jan 2012 23:57:09 -0000
Lewis Neal <email address hidden> wrote:

> Either docutils or leo is looking for html4css1.css1 but gives up too quickly? But why does it find the file with subsequent attempts? I tried many times to reproduce the behaviour without success.

It may not be finding it, not sure if it would be used / needed by the
viewrendered pane.

On Tue, Jan 10, 2012 at 5:57 PM, Lewis Neal <email address hidden> wrote:
> I'm not sure it's an installation problem as the  /lib/site-packages/docutils/writers/html4css1/html4css1.css1 file is present on my system.

Interesting.

> The traceback occurred initially on my leo file and then again on an edited copy to focus on the reSt node. However after the first 2 tracebacks the alt-x vr command then proceeded to work successfully; I did not change anything on my system!

Weird.

> Either docutils or leo is looking for html4css1.css1 but gives up too quickly? But why does it find the file with subsequent attempts? I tried many times to reproduce the behaviour without success.
>
> The docutils installed is the latest snapshot.
> As you seem familiar with the sys.exit problem do you see any value in reporting this to the docutils maintainers as a bug?

Leo has no business looking for .css1 files, and doesn't. This surely
must be a docutils bug. Please do report this to David Goodger.

Edward

I posted at the docutils users mailing list
http://docutils.sourceforge.net/docs/user/mailing-lists.html#docutils-users
Here is a reply from Guenter Milde:
 An error-level system message is issued when the requested file is
 required for the conversion (e.g. to extract data or to embedd it): with
 stylesheets, this means when the "embed-stylesheet" setting is True.

 To avoid sys-exit for missing style sheets, either set embed-stylesheet to
 False or raise the abort-level. I don't know how Leo calls docutils,
 but config files at the standard places should hopefully work.
 http://docutils.sourceforge.net/docs/user/config.html

 > I hope the leo bug link is sufficient for you
 > but please ask if you need any more information or want me to try and
 > reproduce the traceback.

 It would be interesting to to find out, why the standard CSS stylesheet was
 not found in the first place. As a relative path is used, the reason might
 be a "wrong" idea of present work directory.

 Günter

Regards
Lewis

On Sat, Jan 14, 2012 at 2:40 AM, Lewis Neal <email address hidden> wrote:
> I posted at the docutils users mailing list
> http://docutils.sourceforge.net/docs/user/mailing-lists.html#docutils-users
> Here is a reply from Guenter Milde:

Thanks for this.

>  It would be interesting to to find out, why the standard CSS stylesheet was
>  not found in the first place. As a relative path is used, the reason might
>  be a "wrong" idea of present work directory.

This looks like an important clue. I wonder what the "right" idea of
the work directory might be :-)

Hmm. It should be possible to deduce the directories (or set of
directories) for which the relative path should in fact yield the
desired file. I'll look into it.

Edward

P.S. Clearly, we want everything to "just work". That is, we do *not*
want to create yet another rst option as a workaround. If the rst code
does change the working directory, it should restore it when it is
done.

EKR

On Sat, Jan 14, 2012 at 8:18 AM, Edward K. Ream <email address hidden> wrote:

> This looks like an important clue.  I wonder what the "right" idea of
> the work directory might be :-)
>
> Hmm.  It should be possible to deduce the directories (or set of
> directories) for which the relative path should in fact yield the
> desired file.  I'll look into it.

Ah. At last I understand the context. Here (in this bug report), the
call to publish_string happens in the viewrendered plugin, not in
Leo's rst code. In leoRst.py, the one and only call to publish_string
happens in the writeToDocutils method. In the viewrendered plugin,
the one and only call to publish_string happens in the update_rst
method::

    s = publish_string(s,writer_name='html')

As far as the working directory goes, the relevant code (in the plugin) is::

        path = g.scanAllAtPathDirectives(c,p) or c.getNodePath(p)
        if not os.path.isdir(path):
            path = os.path.dirname(path)
        if os.path.isdir(path):
            os.chdir(path)

So the question is, how does docutils create the (weird) relative path::

    '../../../../../lib/site-packages/docutils/writers/html4css1/html4css1.css'

One would think that this path would yield the desired .css file only
if it was invoked five (!) levels below Python's lib directory. In
other words, where do all the "../" come from?

Stay tuned...

EKR

On Sat, Jan 14, 2012 at 8:35 AM, Edward K. Ream <email address hidden> wrote:

> So the question is, how does docutils create the (weird) relative path::
>
>    '../../../../../lib/site-packages/docutils/writers/html4css1/html4css1.css'
>
> One would think that this path would yield the desired .css file only
> if it was invoked five (!) levels below Python's lib directory.  In
> other words, where do all the "../" come from?

I still don't know the answer to the question, but I'm pretty sure I
know where the path is computed and where the error message comes
from.

The message comes from the ctor of the FileInput class in the file
site-customize/docutils/io.py

The actual path,
'../../../../../lib/site-packages/docutils/writers/html4css1/html4css1.css',
most likely comes from the following very strange code at the start of
the Writer class in the file site-customize/docutils/__init__.py:

class Writer(writers.Writer):

    supported = ('html', 'html4css1', 'xhtml')
    """Formats this writer supports."""

    default_stylesheet = 'html4css1.css'

    ### g.pdb('writer/init.py') ### EKR.

    default_stylesheet_path = utils.relative_path(
        os.path.join(os.getcwd(), 'dummy'),
        os.path.join(os.path.dirname(__file__), default_stylesheet))

Note that this code is *not* in the ctor: the default_stylesheet_path
is a class var.

For me, this results in a full, absolute path name. I just now as I
write this see the call to utils.relative_path. Presumably this
creates all the '../' components of the path. Let's see...

def relative_path(source, target):
    """
    Build and return a path to `target`, relative to `source` (both files).

    If there is no common prefix, return the absolute path to `target`.
    """
    source_parts = os.path.abspath(source or 'dummy_file').split(os.sep)
    target_parts = os.path.abspath(target).split(os.sep)
    # Check first 2 parts because '/dir'.split('/') == ['', 'dir']:
    if source_parts[:2] != target_parts[:2]:
        # Nothing in common between paths.
        # Return absolute path, using '/' for URLs:
        return '/'.join(target_parts)
    source_parts.reverse()
    target_parts.reverse()
    while (source_parts and target_parts
           and source_parts[-1] == target_parts[-1]):
        # Remove path components in common:
        source_parts.pop()
        target_parts.pop()
    target_parts.reverse()
    parts = ['..'] * (len(source_parts) - 1) + target_parts
    return '/'.join(parts)

Without understanding the code in detail, the last two lines do indeed
look like the source of the '../' components of the path.

In short, it looks like os.getcwd() is returning a path that results
in utils.relative_path returning
'../../../../../lib/site-packages/docutils/writers/html4css1/html4css1.css'

This is quite bizarre, imo. I wouldn't be surprised if this is a bug
in docutils. All this code seems too clever by half.

The question is, what to do about it.

Edward

On Sun, Jan 15, 2012 at 5:51 AM, Edward K. Ream <email address hidden> wrote:
> In short, it looks like os.getcwd() is returning a path that results
> in utils.relative_path returning
> '../../../../../lib/site-packages/docutils/writers/html4css1/html4css1.css'
>
> This is quite bizarre, imo.  I wouldn't be surprised if this is a bug
> in docutils.  All this code seems too clever by half.
>
> The question is, what to do about it.

Lewis, I would appreciate it if you would ask for comments from the
docutils folks. It seems as though this is really their problem.

Terry, it looks like a workaround would be to save and restore the
curdir in your plugin.

Edward

Edward,

I've been searching the docutils bug list and just moments ago I sent this email to Eli Bendersky about a bug report he made to docutils:

"Eli,

 Did you receive any reply from the docutils group about your Docutils
 bug report? http://article.gmane.org/gmane.text.docutils.user/6112
 I noticed that "(sys.exit) when it can't find a file" is similar to my
 recent bug report http://article.gmane.org/gmane.text.docutils.user/6559

 The creator of leo-editor, Edward Ream has been investigating the issue
 and he may find any replies you received most helpful.
 Please see https://bugs.launchpad.net/leo-editor/+bug/914221

 Thanks
 Lewis"

So the request is not directly to docutils people but I hope Eli may have some helpful comments...

Lewis

On Mon, Jan 16, 2012 at 7:01 AM, Lewis Neal <email address hidden> wrote:

> So the request is not directly to docutils people but I hope Eli may
> have some helpful comments...

Thanks for following up on this matter.

Edward

Here is the reply from Eli
" Hi Lewis,

 Uh, that was a long time ago - I don't recall receiving any meaningful help on that one.

 Eli "

Edward, Here is the second reply from Günter :

On 17.01.12, Lewis Neal wrote:
> Günter,

> Would you mind having a look at Edward's comments at
> https://bugs.launchpad.net/leo-editor/+bug/914221

I did have a look and my response tried to address the issues.

> Many thanks for your assistance with this. I'm happy to provide any
> information if you need it (reproduce the traceback etc...)

> On Fri, Jan 13, 2012, at 09:46 AM, Guenter Milde wrote:
> > On 2012-01-12, Lewis wrote in gmane.text.docutils.user:
> >
> > > To summarise - docutils performs a system exit when it fails to find
> > > the html4css1.css1 file.
> >
> > An error-level system message is issued when the requested file is
> > required for the conversion (e.g. to extract data or to embedd it): with
> > stylesheets, this means when the "embed-stylesheet" setting is True.

OK, testing with

  rst2html --embed-stylesheet --stylesheet=nonexistent.css

I see that no system_message is used and that the --traceback
option does not produce a traceback.

However, as the stylesheet is no essential for further processing, it is
reasonable to skip the embedding, report the error, and continue.
This will be fixed soon.

> > > I hope the leo bug link is sufficient for you
> > > but please ask if you need any more information or want me to try and
> > > reproduce the traceback.

> > It would be interesting to to find out, why the standard CSS
> > stylesheet was not found in the first place. As a relative path is
> > used, the reason might be a wrong idea of present work directory.

It seems like a configuration problem: when the stylesheet and
stylesheet-path options are not set, the default is used. The actual path
depends on the installation and is an absolute path.

The relative path in the error message suggest that this path was passed as
config option (via config file, command-line argument, or programatically)
to Docutils. In this case, it must be relative to the working directory, not
the input document.

mfg

Günter
======

I also cc'd David Goodger on the email to Günter.

Regards
Lewis

On Wed, Jan 18, 2012 at 4:42 AM, Lewis Neal <email address hidden> wrote:
> Edward, Here is the second reply from Günter :

Thanks for keeping us all informed.

Edward

Günter has replied again on 20 Jan 10:50 - see http://permalink.gmane.org/gmane.text.docutils.user/6565

For convenience here is a copy of the text:
Re: No such file or directory: html4css1.css - system exit
Guenter Milde <milde <at> users.sf.net>
2012-01-20 09:50:34 GMT

On 2012-01-12, Lewis wrote:

> You are most likely aware that the Leo-editor project makes use of Docutils.

I added Leo to the Docutils link list.

> To summarise - docutils performs a system exit when it fails to find the
> html4css1.css1 file.

For style files, this is fixed in SVN.

For the generic behaviour of `docutils.io.FileInput`, I'll respond to the
old request you pointed to (join docutils-users or watch the list via gmane
or another archive if you want to follow the discussion).

> I hope the leo bug link is sufficient for you but please ask if you
> need any more information or want me to try and reproduce the
> traceback.

I Leo calling docutils programatically?

I found a possible reason of the failure to find the html4css1 file:

Leos tutorial
http://webpages.charter.net/edreamleo/rstplugin3.html#step-1-create-the-rst-node

suggests:

  Here are the recommended options when using docutils:

  @ @rst-options
  call_docutils=True
  code_mode=False
  stylesheet_path=..\doc
  write_intermediate_file=True
  @c

However, unless a user is using custom style sheets, it is better to keep the
Docutils stylesheet options at the default value.

A very confusing point is that

* for Docutils, `stylesheet_path` is a path (or a list of paths) to a
  stylesheet *file*:

  --stylesheet=<URL> Specify comma separated list of stylesheet URLs.
                        Overrides previous --stylesheet and --stylesheet-path
                        settings.
  --stylesheet-path=<file>
                        Specify comma separated list of stylesheet paths. With
                        --link-stylesheet, the path is rewritten relative to
                        the output file. Default:
                        "/<docutils-package-dir>/docutils/writers/html4css1/html4css1.css"

* for Leo, `stylesheet_path` points to a *directory* of stylesheets:

  stylesheet_name (default: ‘default.css’)

      The name of the stylesheet passed to docutils.

  stylesheet_path (default: ‘’)

      The directory containing the stylesheet passed to docutils.

      Note: If the stylesheet_embed option is True, specify a path
      relative to the location of the Leo file. If the stylesheet_embed
      option is False, specify a path relative to the location of the
      HTML file.

Also:

* `default.css` is outdated and specific to HTML.
  Using Docutils' default value for (Docutils) stylesheet_path option
  ensures an up-to-date style sheet in a location Docutils can find
  out-of-the-box.

* Leo's "stylesheet_path" option would not need to depend on the
  stylesheet_embed setting, with the translation:

     Leo Docutils

     stylesheet_path+stylesheet_name --> stylesheet_path

  instead of

     stylesheet_path+stylesheet_name --> stylesheet

Thanks,

Günter

I'm marking this as "fix released" so it no longer appears in searches.
A link to this bug now appears here: https://github.com/leo-editor/leo-editor/issues/73